Mini Course in Deep Learning with PyTorch for AIMS

The African Masters of Machine Intelligence (AMMI) is Africa's flagship program in machine intelligence led by The African Institute for Mathematical Sciences (AIMS). These lessons, developed during the course of several years while I've been teaching at Purdue and NYU, are here proposed for the AMMI (AIMS).

Prior to this course delivered for AMMI (AIMS), an earlier version of this was delivered and video-recorded for the Computational and Data Science for High Energy Physics (CoDaS-HEP) summer school at Princeton University. Please refer to this version release here.

Table of contents

T: theory (slides and animations)
P: practice (Jupyter Notebooks)

1. T Learning paradigms: supervised-, unsupervised-, and reinforcement-learning
2. P Getting started with the tools: Jupyter notebook, PyTorch tensors and autodifferentiation
3. T+P Neural net's forward and backward propagation for classification and regression
4. T+P Convolutional neural nets improve performance by exploiting data nature
5. T+P Foundations of Salsa
6. T+P Recurrent nets natively support sequential data
7. T+P Unsupervised learning: vanilla and variational autoencoders, generative adversarial nets
8. T How to create and deliver an effective presentation
9. T+P Regularization for neural nets

Sessions and relative material

1. Time slot 1 (4h + 4h)
   Topics: 1, 2, 3.
   Slides: 01 - ML and spiral classification.
   Notebooks: 01, 02, 03, 04, 05.
2. Time slot 2 (4h + 2h)
   Topic: 4.
   Slides: 02 - CNN.
   Notebooks: 06, 07.
3. Time slot 3 (2h)
   Topic: 5.
   Slides: 03 - Salsa.
4. Time slot 4 (4h + 4h)
   Topic: 6.
   Slides: 04 - RNN.
   Code Readings: Word Language Model.
   Assignment: HW1, HW1 Solutions.
   Notebooks: 08, 09.
5. Time slot 5 (4h + 4h)
   Topic: 7.
   Slides: 05 - Generative models.
   Code Readings: GAN.
   Guides: TikZ Quick Guide.
   Notebooks: 10, 11.
6. Time slot 6 (1h)
   Topic: 8.
   Slides: 06 - How to present.
   Video: How to prepare a presentation.
7. Time slot 7 (1h)
   Topic: 9.
   Slides: 07 - Regularisation.
   Assignment: HW2.
   Notebooks: 12.

Notebooks visualisation

Jupyter Notebooks are used throughout these lectures for interactive data exploration and visualisation.

We use dark styles for both GitHub and Jupyter Notebook. You should try to do the same, or they will look ugly. JupyterLab has a built-in selectable dark theme, so you only need to install something if you want to use the classic notebook interface. To see the content appropriately in the classic interface install the following:

• Jupyter Notebook dark theme;
• GitHub dark theme and comment out the invert #fff to #181818 code block.

Keeping in touch

Feel free to follow Afredo at Twitter and subscribe to his YouTube channel to have the latest free educational material.

For more educational materials you also can head to Ritchie's website.

Getting started

To be able to follow the workshop exercises, you are going to need a laptop with Miniconda (a minimal version of Anaconda) and several Python packages installed. Following instruction would work as is for Mac or Ubuntu linux users, Windows users would need to install and work in the Gitbash terminal.

Download and install Miniconda

Please go to the Anaconda website. Download and install the latest Miniconda version for Python 3.6 for your operating system.

wget <http:// link to miniconda>
sh <miniconda .sh>

After that, type:

conda --help

and read the manual.

Check-out the git repository with the exercise

Once Miniconda is ready, checkout the course repository and and proceed with setting up the environment:

git clone https://github.com/Atcold/PyTorch-Deep-Learning-Minicourse

If you do not have git and do not wish to install it, just download the repository as zip, and unpack it:

wget https://github.com/Atcold/PyTorch-Deep-Learning-Minicourse/archive/master.zip
#For Mac users:
#curl -O https://github.com/Atcold/PyTorch-Deep-Learning-Minicourse/archive/master.zip
unzip master.zip

Create isolated Miniconda environment

Change into the course folder, then type:

#cd PyTorch-Deep-Learning-Minicourse
conda env create -f environment.yml
source activate dl-minicourse

Install Autocomplete by hinterland

You have to run the following commands if you want auto-complete.

pip install jupyter_contrib_nbextensions
pip install jupyter_nbextensions_configurator
jupyter contrib nbextension install --user

cd /usr/local/miniconda3/envs/dl-minicourse/lib/python3.6/site-packages/jupyter_contrib_nbextensions/nbextensions
jupyter nbextension install hinterland
jupyter nbextension enable hinterland/hinterland

Start Jupyter notebook or JupyterLab

If you are working in a JupyterLab container double click on "Files" tab in the upper right corner. Locate first notebook, double click to open. Do not attempt to start jupyter from the terminal window.

If working on a laptop, start from terminal as usual:

jupyter lab

Or, for the classic interface:

jupyter notebook

More PyTorch Resources

If you would like more PyTorch resources, head over to the global community-maintained repository of hundreds of reliable implementations and guides at the following repository created by Ritchie Ng: The Incredible PyTorch.