diff --git a/mkdocs/docs/HPC/FAQ.md b/mkdocs/docs/HPC/FAQ.md index 4e70ff712b7..4e9ac13b033 100644 --- a/mkdocs/docs/HPC/FAQ.md +++ b/mkdocs/docs/HPC/FAQ.md @@ -319,6 +319,12 @@ Please send an e-mail to {{hpcinfo}} that includes: {% endif %} +If the software is a Python package, you can manually install it in a virtual environment. +More information can be found [here](./setting_up_python_virtual_environments.md). +Note that it is still preferred to submit a software installation request, +as the software installed by the HPC team will be optimized for the HPC environment. +This can lead to dramatic performance improvements. + ### Is my connection compromised? Remote host identification has changed On Monday 25 April 2022, the login nodes received an update to RHEL8. diff --git a/mkdocs/docs/HPC/setting_up_python_virtual_environments.md b/mkdocs/docs/HPC/setting_up_python_virtual_environments.md index e73b59bd0fd..1d228c44b05 100644 --- a/mkdocs/docs/HPC/setting_up_python_virtual_environments.md +++ b/mkdocs/docs/HPC/setting_up_python_virtual_environments.md @@ -25,6 +25,14 @@ python -m venv myenv # Create a new virtual environment named 'myenv' This command creates a new subdirectory named `myenv` in the current working directory. This directory will contain the packages, scripts, and binaries that are needed to manage the virtual environment. +!!! warning + When you create a virtual environment on top of a loaded Python module, + the environment becomes specific to the cluster you're working on. + This is because modules are built and optimized for the operating system and CPUs of the cluster. + This means that you should create a new virtual environment for each cluster you work on. + See [Creating a virtual environment for a specific cluster](#creating-a-virtual-environment-for-a-specific-cluster) for more information. + + ### Activating a virtual environment To use the virtual environment, you need to *activate* it. @@ -49,10 +57,18 @@ No administrator privileges are required to install packages in a virtual enviro It is now possible to run Python scripts that use the installed packages in the virtual environment. -!!! note - Always prefer to use Python packages that are centrally installed, which are available via the environment modules interface. - Software installations available via the modules interface are compiled and optimized for the HPC cluster, while packages installed with `pip` are not. - To check if a package is available as a module, you can use the following command: +!!! tip + When creating a virtual environment, it's best to install only *pure* Python + packages. Pure Python packages consist solely of Python code and don't require compilation. + The installation method of these packages doesn't impact performance since they're not compiled. + + Compiled libraries with a Python wrapper (*non-pure* Python packages) are better loaded as modules + rather than installed in the virtual environment. + This is because modules are optimized for the HPC cluster’s specific hardware and operating system. + If a non-pure Python package isn't available as a module, you can + [submit a software installation request](https://www.ugent.be/hpc/en/support/software-installation-request). + + To check if a package is available as a module, use: ```bash module av package_name