Merge branch 'main' into detect-html-workflow

hpcugent · Sep 19, 2024 · 9c67553 · 9c67553
2 parents ddf573c + 35ab523
commit 9c67553
Show file tree

Hide file tree

Showing 118 changed files with 1,819 additions and 172 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -10,8 +10,6 @@ jobs:
     steps:
       - name: checkout
         uses: actions/checkout@v3
-        with:
-          fetch-depth: 0
 
       - name: set up Python
         uses: actions/setup-python@v4

diff --git a/config/templates/hpc.template b/config/templates/hpc.template
@@ -14,6 +14,9 @@ nav:
   - Running batch jobs: running_batch_jobs.md
   - Running interactive jobs: running_interactive_jobs.md
   - Running jobs with input/output data: running_jobs_with_input_output_data.md
+{%- if site == 'Gent' %}
+  - Available software: ./only/gent/available_software/index.md
+{%- endif %}
   - Multi core jobs/Parallel Computing: multi_core_jobs.md
   - Using the HPC-UGent web portal: web_portal.md
 {%- if site == 'Gent' %}
@@ -54,6 +57,7 @@ nav:
       - MATLAB: MATLAB.md
       - mympirun: mympirun.md
       - OpenFOAM: openFOAM.md
+      - Python: python.md
       - Python virtual environments: setting_up_python_virtual_environments.md
   - FAQ:
       - Frequently Asked Questions: FAQ.md

diff --git a/mkdocs/docs/HPC/FAQ.md b/mkdocs/docs/HPC/FAQ.md
@@ -325,6 +325,18 @@ Note that it is still preferred to submit a software installation request,
 as the software installed by the HPC team will be optimized for the HPC environment.
 This can lead to dramatic performance improvements.
 
+!!! warning
+    Do not use `pip install --user` to install software, as this places packages 
+    in `~/.local/lib/python*/site-packages`, potentially causing conflicts with software installed by the HPC team. 
+
+    Also, avoid using `pip install --prefix` or `pip install --target` with `$PYTHONPATH`, 
+    since `$PYTHONPATH` is a global setting that affects all Python versions.
+    When `$PYTHONPATH` includes directories with packages from different Python versions, it can lead to conflicts. 
+
+    Instead, use a Python virtual environment for package installations in your home directory. 
+    For guidance on setting up virtual environments, 
+    refer to the [Python virtual environments setup guide](./setting_up_python_virtual_environments.md).
+
 ### Is my connection compromised? Remote host identification has changed
 
 On Monday 25 April 2022, the login nodes received an update to RHEL8.

diff --git a/mkdocs/docs/HPC/apptainer.md b/mkdocs/docs/HPC/apptainer.md
@@ -16,7 +16,7 @@ For more general information about the use of Apptainer, please see the
 official documentation at <https://apptainer.org/docs/>.
 
 This documentation only covers aspects of using Apptainer on the
-{{hpcinfra}} infrastructure.
+{{hpcinfra}}.
 
 ## Restrictions on image location
 
@@ -46,26 +46,20 @@ filesystems, and they will be accessible via the familiar `$VSC_HOME`,
 
 ### Creating Apptainer/Singularity images
 
-Creating new Apptainer/Singularity images or converting Docker images,
-by default, requires admin privileges, which is obviously not available
-on the {{hpcinfra}} infrastructure. However, if you use the `--fakeroot` option, you
-can make new Apptainer/Singularity images or convert Docker images.
-
-Due to the nature of `--fakeroot` option, we recommend to write your
-Apptainer/Singularity image to a globally writable location, like
-`/tmp`, or `/local` directories. Once the image is created, you should
-move it to your desired destination. An example to make an
-Apptainer/Singularity container image:
+We recommend writing your Apptainer/Singularity image to a globally writable location, 
+like `/tmp`, or `/local` directories. 
+Once the image is created, you should move it to your desired destination. 
+An example to make an Apptainer/Singularity container image:
 
 ```shell
 # avoid that Apptainer uses $HOME/.cache
 export APPTAINER_CACHEDIR=/tmp/$USER/apptainer/cache
-# instruct Apptainer to use temp dir on local filessytem
+# instruct Apptainer to use temp dir on local filesystem
 export APPTAINER_TMPDIR=/tmp/$USER/apptainer/tmpdir
 # specified temp dir must exist, so create it
 mkdir -p $APPTAINER_TMPDIR
 # convert Docker container to Apptainer container image
-apptainer build --fakeroot /tmp/$USER/tf.sif docker://nvcr.io/nvidia/tensorflow:21.10-tf1-py3
+apptainer build /tmp/$USER/tf.sif docker://nvcr.io/nvidia/tensorflow:21.10-tf1-py3
 # mv container image to $VSC_SCRATCH
 mv /tmp/$USER/tf.sif $VSC_SCRATCH/tf.sif
 ```
@@ -114,7 +108,7 @@ factor 1234567
 ## Tensorflow example
 
 We already have a Tensorflow example image, but you can also convert the
-Docker image (see <https://hub.docker.com/r/tensorflow/tensorflow>) to a
+Docker image (see <https://hub.docker.com/r/tensorflow/tensorflow>) to an
 Apptainer/Singularity image yourself
 
 Copy testing image from `/apps/gent/tutorials` to `$VSC_SCRATCH`:

diff --git a/mkdocs/docs/HPC/getting_started.md b/mkdocs/docs/HPC/getting_started.md
@@ -142,7 +142,7 @@ Our job script looks like this:
 ```bash title="run.sh"
 #!/bin/bash
 
-module load TensorFlow/2.11.0-foss-2022a
+module load TensorFlow/2.15.1-foss-2023a
 
 python tensorflow_mnist.py
 ```

diff --git a/mkdocs/docs/HPC/img/ood_desktop_running.png b/mkdocs/docs/HPC/img/ood_desktop_running.png
diff --git a/mkdocs/docs/HPC/img/ood_file_explorer.png b/mkdocs/docs/HPC/img/ood_file_explorer.png
diff --git a/mkdocs/docs/HPC/img/ood_hamburger.png b/mkdocs/docs/HPC/img/ood_hamburger.png
diff --git a/mkdocs/docs/HPC/img/ood_jupyter_open_shell.png b/mkdocs/docs/HPC/img/ood_jupyter_open_shell.png
diff --git a/mkdocs/docs/HPC/img/ood_launch_desktop.png b/mkdocs/docs/HPC/img/ood_launch_desktop.png
diff --git a/mkdocs/docs/HPC/img/ood_open_shell.png b/mkdocs/docs/HPC/img/ood_open_shell.png
diff --git a/mkdocs/docs/HPC/img/ood_relaunch_desktop_session.png b/mkdocs/docs/HPC/img/ood_relaunch_desktop_session.png
diff --git a/mkdocs/docs/HPC/img/ood_start.png b/mkdocs/docs/HPC/img/ood_start.png
diff --git a/mkdocs/docs/HPC/index.md b/mkdocs/docs/HPC/index.md
@@ -21,6 +21,7 @@ Use the OS dropdown in the top bar to switch to a different operating system.
 - [Recording of HPC-UGent intro](https://www.ugent.be/hpc/en/training/introhpcugent-recording)
 - [Linux Tutorial](linux-tutorial/index.md)
 - [Hardware overview](https://www.ugent.be/hpc/en/infrastructure)
+- [Available software](./only/gent/available_software/index.md)
 - [Migration of cluster and login nodes to RHEL9 (starting Sept'24)](rhel9.md)
 {%- endif %}
 - [FAQ](FAQ.md) | [Troubleshooting](troubleshooting.md) | [Best practices](./best_practices.md) | [Known issues](known_issues.md)

diff --git a/mkdocs/docs/HPC/jupyter.md b/mkdocs/docs/HPC/jupyter.md
@@ -45,9 +45,7 @@ The first thing we need to do is finding the modules that contain our package of
 
 To find the appropriate modules, it is recommended to use the shell within the web portal under `Clusters`>`>_login Shell Access`.
 
-
-![image](img/ood_jupyter_open_shell.png)
-
+![image](img/ood_open_shell.png)
 
 We can see all available versions of the SciPy module by using `module avail SciPy-bundle`:
 

diff --git a/mkdocs/docs/HPC/linux-tutorial/hpc_infrastructure.md b/mkdocs/docs/HPC/linux-tutorial/hpc_infrastructure.md
@@ -67,7 +67,7 @@ Python, and prints the numbers to a *unique* output file in
 Hint: `python -c "print(sum(range(1, 101)))"`
 
 -   How many modules are available for Python version 3.6.4?
--   How many modules get loaded when you load the `Python/3.6.4-intel-2018a` module?
+-   How many modules get loaded when you load the `Python/3.12.3-GCCcore-13.3.0` module?
 -   Which `cluster` modules are available?
 
 -   What's the full path to your personal home/data/scratch directories?

diff --git a/mkdocs/docs/HPC/multi_job_submission.md b/mkdocs/docs/HPC/multi_job_submission.md
@@ -136,7 +136,7 @@ The job can now be submitted as follows (to check which `worker` module
 to use, see subsection [Using explicit version numbers](running_batch_jobs.md#using-explicit-version-numbers)):
 
 ```
-$ module load worker/1.6.12-foss-2021b
+$ module load worker/1.6.13-iimpi-2023a
 $ wsub -batch weather.pbs -data data.csv
 total number of work items: 41
 {{jobid}}
@@ -273,7 +273,7 @@ Note that
 The job is now submitted as follows:
 
 ```
-$ module load worker/1.6.12-foss-2021b
+$ module load worker/1.6.13-iimpi-2023a
 $ wsub -t 1-100 -batch test_set.pbs
 total number of work items: 100
 {{jobid}}

diff --git a/mkdocs/docs/HPC/only/gent/available_software/data/json_data.json b/mkdocs/docs/HPC/only/gent/available_software/data/json_data.json
diff --git a/mkdocs/docs/HPC/only/gent/available_software/data/json_data_detail.json b/mkdocs/docs/HPC/only/gent/available_software/data/json_data_detail.json
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/4ti2.md b/mkdocs/docs/HPC/only/gent/available_software/detail/4ti2.md
@@ -21,4 +21,4 @@ module load 4ti2/1.6.10-GCC-13.2.0
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-|4ti2/1.6.10-GCC-13.2.0|x|-|x|-|x|-|-|
+|4ti2/1.6.10-GCC-13.2.0|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/BRiAl.md b/mkdocs/docs/HPC/only/gent/available_software/detail/BRiAl.md
@@ -0,0 +1,24 @@
+---
+hide:
+  - toc
+---
+
+BRiAl
+=====
+
+# Available modules
+
+
+The overview below shows which BRiAl installations are available per HPC-UGent Tier-2 cluster, ordered based on software version (new to old).
+
+To start using BRiAl, load one of these modules using a `module load` command like:
+
+```shell
+module load BRiAl/1.2.12-GCC-13.2.0
+```
+
+*(This data was automatically generated on {{modules_last_updated}})*  
+
+| |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
+| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|BRiAl/1.2.12-GCC-13.2.0|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/BiG-SCAPE.md b/mkdocs/docs/HPC/only/gent/available_software/detail/BiG-SCAPE.md
@@ -14,11 +14,12 @@ The overview below shows which BiG-SCAPE installations are available per HPC-UGe
 To start using BiG-SCAPE, load one of these modules using a `module load` command like:
 
 ```shell
-module load BiG-SCAPE/1.0.1-intel-2019b-Python-3.7.4
+module load BiG-SCAPE/1.1.9-foss-2023b
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|BiG-SCAPE/1.1.9-foss-2023b|x|x|x|x|x|x|x|
 |BiG-SCAPE/1.0.1-intel-2019b-Python-3.7.4|-|x|x|-|x|-|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/Biopython.md b/mkdocs/docs/HPC/only/gent/available_software/detail/Biopython.md
@@ -14,13 +14,14 @@ The overview below shows which Biopython installations are available per HPC-UGe
 To start using Biopython, load one of these modules using a `module load` command like:
 
 ```shell
-module load Biopython/1.83-foss-2023a
+module load Biopython/1.84-foss-2023b
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|Biopython/1.84-foss-2023b|x|x|x|x|x|x|x|
 |Biopython/1.83-foss-2023a|x|x|x|x|x|x|x|
 |Biopython/1.81-foss-2022b|x|x|x|x|x|-|x|
 |Biopython/1.79-foss-2022a|x|x|x|x|x|x|x|

diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/Boost.MPI.md b/mkdocs/docs/HPC/only/gent/available_software/detail/Boost.MPI.md
@@ -14,13 +14,14 @@ The overview below shows which Boost.MPI installations are available per HPC-UGe
 To start using Boost.MPI, load one of these modules using a `module load` command like:
 
 ```shell
-module load Boost.MPI/1.81.0-gompi-2022b
+module load Boost.MPI/1.82.0-gompi-2023a
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|Boost.MPI/1.82.0-gompi-2023a|x|x|x|x|x|x|x|
 |Boost.MPI/1.81.0-gompi-2022b|x|x|x|x|x|-|x|
 |Boost.MPI/1.79.0-gompi-2022a|-|x|x|x|x|-|x|
 |Boost.MPI/1.77.0-gompi-2021b|x|x|x|-|x|-|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/CLHEP.md b/mkdocs/docs/HPC/only/gent/available_software/detail/CLHEP.md
@@ -14,13 +14,14 @@ The overview below shows which CLHEP installations are available per HPC-UGent T
 To start using CLHEP, load one of these modules using a `module load` command like:
 
 ```shell
-module load CLHEP/2.4.6.4-GCC-12.2.0
+module load CLHEP/2.4.7.1-GCC-12.3.0
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|CLHEP/2.4.7.1-GCC-12.3.0|x|x|x|x|x|x|x|
 |CLHEP/2.4.6.4-GCC-12.2.0|x|x|x|x|x|-|x|
 |CLHEP/2.4.5.3-GCC-11.3.0|x|x|x|x|x|-|x|
 |CLHEP/2.4.5.1-GCC-11.2.0|x|x|x|x|x|-|x|

diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/CUDD.md b/mkdocs/docs/HPC/only/gent/available_software/detail/CUDD.md
@@ -0,0 +1,24 @@
+---
+hide:
+  - toc
+---
+
+CUDD
+====
+
+# Available modules
+
+
+The overview below shows which CUDD installations are available per HPC-UGent Tier-2 cluster, ordered based on software version (new to old).
+
+To start using CUDD, load one of these modules using a `module load` command like:
+
+```shell
+module load CUDD/3.0.0-GCC-13.2.0
+```
+
+*(This data was automatically generated on {{modules_last_updated}})*  
+
+| |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
+| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|CUDD/3.0.0-GCC-13.2.0|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/CoCoALib.md b/mkdocs/docs/HPC/only/gent/available_software/detail/CoCoALib.md
@@ -0,0 +1,24 @@
+---
+hide:
+  - toc
+---
+
+CoCoALib
+========
+
+# Available modules
+
+
+The overview below shows which CoCoALib installations are available per HPC-UGent Tier-2 cluster, ordered based on software version (new to old).
+
+To start using CoCoALib, load one of these modules using a `module load` command like:
+
+```shell
+module load CoCoALib/0.99850-GCC-13.2.0
+```
+
+*(This data was automatically generated on {{modules_last_updated}})*  
+
+| |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
+| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|CoCoALib/0.99850-GCC-13.2.0|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/Cython.md b/mkdocs/docs/HPC/only/gent/available_software/detail/Cython.md
@@ -22,7 +22,7 @@ module load Cython/3.0.10-GCCcore-13.3.0
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 |Cython/3.0.10-GCCcore-13.3.0|x|x|x|x|x|x|x|
-|Cython/3.0.10-GCCcore-13.2.0|x|x|x|-|x|x|-|
+|Cython/3.0.10-GCCcore-13.2.0|x|x|x|x|x|x|x|
 |Cython/3.0.8-GCCcore-12.3.0|x|x|x|x|x|x|x|
 |Cython/3.0.8-GCCcore-12.2.0|x|x|x|x|x|-|x|
 |Cython/3.0.7-GCCcore-12.3.0|x|x|x|x|x|-|x|

diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/E-ANTIC.md b/mkdocs/docs/HPC/only/gent/available_software/detail/E-ANTIC.md
@@ -0,0 +1,24 @@
+---
+hide:
+  - toc
+---
+
+E-ANTIC
+=======
+
+# Available modules
+
+
+The overview below shows which E-ANTIC installations are available per HPC-UGent Tier-2 cluster, ordered based on software version (new to old).
+
+To start using E-ANTIC, load one of these modules using a `module load` command like:
+
+```shell
+module load E-ANTIC/2.0.2-gfbf-2023b
+```
+
+*(This data was automatically generated on {{modules_last_updated}})*  
+
+| |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
+| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|E-ANTIC/2.0.2-gfbf-2023b|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/ECL.md b/mkdocs/docs/HPC/only/gent/available_software/detail/ECL.md
@@ -21,4 +21,4 @@ module load ECL/24.5.10-GCCcore-13.2.0
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-|ECL/24.5.10-GCCcore-13.2.0|x|x|x|-|x|x|-|
+|ECL/24.5.10-GCCcore-13.2.0|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/FFLAS-FFPACK.md b/mkdocs/docs/HPC/only/gent/available_software/detail/FFLAS-FFPACK.md
@@ -21,4 +21,4 @@ module load FFLAS-FFPACK/2.5.0-gfbf-2023b
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-|FFLAS-FFPACK/2.5.0-gfbf-2023b|x|x|x|-|x|x|-|
+|FFLAS-FFPACK/2.5.0-gfbf-2023b|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/FLINT.md b/mkdocs/docs/HPC/only/gent/available_software/detail/FLINT.md
@@ -21,4 +21,4 @@ module load FLINT/3.1.1-gfbf-2023b
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-|FLINT/3.1.1-gfbf-2023b|x|x|x|x|x|-|x|
+|FLINT/3.1.1-gfbf-2023b|x|x|x|x|x|x|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/FLTK.md b/mkdocs/docs/HPC/only/gent/available_software/detail/FLTK.md
@@ -14,12 +14,13 @@ The overview below shows which FLTK installations are available per HPC-UGent Ti
 To start using FLTK, load one of these modules using a `module load` command like:
 
 ```shell
-module load FLTK/1.3.5-GCCcore-10.2.0
+module load FLTK/1.3.9-GCCcore-13.2.0
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|FLTK/1.3.9-GCCcore-13.2.0|x|x|x|x|x|x|x|
 |FLTK/1.3.5-GCCcore-10.2.0|-|x|x|x|x|-|x|
 |FLTK/1.3.5-GCCcore-9.3.0|-|x|x|-|x|-|x|
diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/FastTree.md b/mkdocs/docs/HPC/only/gent/available_software/detail/FastTree.md
@@ -14,13 +14,14 @@ The overview below shows which FastTree installations are available per HPC-UGen
 To start using FastTree, load one of these modules using a `module load` command like:
 
 ```shell
-module load FastTree/2.1.11-GCCcore-12.3.0
+module load FastTree/2.1.11-GCCcore-13.2.0
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|FastTree/2.1.11-GCCcore-13.2.0|x|x|x|x|x|x|x|
 |FastTree/2.1.11-GCCcore-12.3.0|x|x|x|x|x|x|x|
 |FastTree/2.1.11-GCCcore-11.3.0|x|x|x|x|x|-|x|
 |FastTree/2.1.11-GCCcore-11.2.0|x|x|x|-|x|-|x|

diff --git a/mkdocs/docs/HPC/only/gent/available_software/detail/GATE.md b/mkdocs/docs/HPC/only/gent/available_software/detail/GATE.md
@@ -14,13 +14,14 @@ The overview below shows which GATE installations are available per HPC-UGent Ti
 To start using GATE, load one of these modules using a `module load` command like:
 
 ```shell
-module load GATE/9.2-foss-2022a
+module load GATE/9.4-foss-2023a
 ```
 
 *(This data was automatically generated on {{modules_last_updated}})*  
 
 | |accelgor|doduo|donphan|gallade|joltik|shinx|skitty|
 | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|GATE/9.4-foss-2023a|x|x|x|x|x|x|x|
 |GATE/9.2-foss-2022a|x|x|x|x|x|-|x|
 |GATE/9.2-foss-2021b|x|x|x|x|x|-|x|
 |GATE/9.1-foss-2021b|x|x|x|x|x|-|x|