Skip to content

Commit

Permalink
Merge pull request #673 from lbarraga/venv-docs
Browse files Browse the repository at this point in the history
Add tip and warning to venv docs, add link in FAQ
  • Loading branch information
boegel authored Aug 29, 2024
2 parents 35d3ca5 + c15a069 commit 1da87a6
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 4 deletions.
6 changes: 6 additions & 0 deletions mkdocs/docs/HPC/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,12 @@ Please send an e-mail to {{hpcinfo}} that includes:

{% endif %}

If the software is a Python package, you can manually install it in a virtual environment.
More information can be found [here](./setting_up_python_virtual_environments.md).
Note that it is still preferred to submit a software installation request,
as the software installed by the HPC team will be optimized for the HPC environment.
This can lead to dramatic performance improvements.

### Is my connection compromised? Remote host identification has changed

On Monday 25 April 2022, the login nodes received an update to RHEL8.
Expand Down
24 changes: 20 additions & 4 deletions mkdocs/docs/HPC/setting_up_python_virtual_environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,14 @@ python -m venv myenv # Create a new virtual environment named 'myenv'
This command creates a new subdirectory named `myenv` in the current working directory.
This directory will contain the packages, scripts, and binaries that are needed to manage the virtual environment.

!!! warning
When you create a virtual environment on top of a loaded Python module,
the environment becomes specific to the cluster you're working on.
This is because modules are built and optimized for the operating system and CPUs of the cluster.
This means that you should create a new virtual environment for each cluster you work on.
See [Creating a virtual environment for a specific cluster](#creating-a-virtual-environment-for-a-specific-cluster) for more information.


### Activating a virtual environment

To use the virtual environment, you need to *activate* it.
Expand All @@ -49,10 +57,18 @@ No administrator privileges are required to install packages in a virtual enviro

It is now possible to run Python scripts that use the installed packages in the virtual environment.

!!! note
Always prefer to use Python packages that are centrally installed, which are available via the environment modules interface.
Software installations available via the modules interface are compiled and optimized for the HPC cluster, while packages installed with `pip` are not.
To check if a package is available as a module, you can use the following command:
!!! tip
When creating a virtual environment, it's best to install only *pure* Python
packages. Pure Python packages consist solely of Python code and don't require compilation.
The installation method of these packages doesn't impact performance since they're not compiled.

Compiled libraries with a Python wrapper (*non-pure* Python packages) are better loaded as modules
rather than installed in the virtual environment.
This is because modules are optimized for the HPC cluster’s specific hardware and operating system.
If a non-pure Python package isn't available as a module, you can
[submit a software installation request](https://www.ugent.be/hpc/en/support/software-installation-request).

To check if a package is available as a module, use:

```bash
module av package_name
Expand Down

0 comments on commit 1da87a6

Please sign in to comment.