Getting started with Béluga

Virtual environments

(a) Creating a virtual environment

A virtual environment can be created using virtualenv ~/virtualenvs/Uvenv

(Here, ~/virtualenvs/Uvenv is the path to the virtual environment)

Note: If a specific python version is required for your work, you could load that version [e.g. module load python/3.6.3] before creating the virtual environment.

(b) Activating a virtual environment

source ~/virtualenvs/Uvenv/bin/activate

(c) Deactivating the environment

The environment can be deactivated at anytime using: deactivate

Interactive environments

Before writing a script for submitting a task, testing and debugging is usually done as an interactive job. It will be easier to transition to a job script once your work is debugged and tested.

To use the GPU in an interactive job, we need to access a node and request for resources

salloc --time=1:0:0 --cpus-per-task=8 --mem 32000M --gres=gpu:1 --account=def-username-xx

Note: you need to change the username accordingly in: --account=def-username-xx

More information can be found here: https://docs.computecanada.ca/wiki/Running_jobs#Interactive_jobs

Managing jobs

Information on different types of jobs can be found here: https://docs.computecanada.ca/wiki/Running_jobs#Examples_of_job_scripts

(a) Submitting a job

When GPUs are required for a job (e.g. deep learning application), you need to pay attention to CPUs and memory allocation per node when requesting for resources. An overview of hardware available for Compute Canada clusters and using GPUs with SLURM can be found here: https://docs.computecanada.ca/wiki/Using_GPUs_with_Slurm

An example for a script for a job needing GPUs is given below. You may need to change the parameters accordingly:

#!/bin/bash
#SBATCH --gres=gpu:2        # Request GPU "generic resources"
#SBATCH --cpus-per-task=24  # Cores proportional to GPUs: 6 on Cedar, 10 on Béluga, 16 on Graham.
#SBATCH --mem=45G           # Memory proportional to GPUs: 32000 Cedar, 47000 Béluga, 64000 Graham.
#SBATCH --time=0-03:00:00   # DD-HH:MM:SS
#SBATCH --job-name=UMAGeT-norm
#SBATCH --account=def-username-xx
#SBATCH --chdir=/path/to/project_directory/
#SBATCH [email protected]
#SBATCH --mail-type=BEGIN
#SBATCH --mail-type=END

source /path/to/virtual_environment/bin/activate
python main.py 

(b) Job dependencies

In the event that one of your jobs need to be complete for another job to begin, you may submit the job as follows:

JOB1_ID=$(sbatch --parsable job_1.sh)
sbatch --dependency=afterok:$JOB1_ID job_2.sh