| Topic 💥 | Description 📘 |
|---|---|
JAX, Optax, Haiku and CNN  |
This tutorial covers the prerequisites for the rest of the practicals: JAX, Optax and Haiku are high performance libraries for computation, optimization and model learning. We then introduce Convolutional Neural Networks (CNN), and show how to train a deep convolutional model using the aforementioned tools. |
| Transformers and Attention |
Attention is a mechanism whereby a model can focus on parts of the input data that are relevant to perform its inference. The Transformer architecture Vaswani et al. 2017 provides an effective and versatile method to implement the concept of Attention in Deep Networks. Arguably, Transformes have become the de-facto architecture for complex Natural Language Processing (NLP) tasks, but can can also be applied in various domains reaching state-of-the-art performance, including computer vision and reinforcement learning. In this practical, we will introduce attention in greater detail and build the entire transformer architecture block by block. |
| Graph Neural Networks |
In this tutorial, we will be learning about Graph Neural Networks (GNNs), a topic which has exploded in popularity in both research and industry. We will start with a refresher on graph theory, then dive into how GNNs work from a high level. Next we will cover some popular GNN implementations and see how they work in practice. |
| Deep Generative Models |
Generative models are unsupervised methods which whose purpose is to generate new samples akin to those seen in the training dataset. Deep learning has taken these models to a new level with the introduction of architectures such as GANs, VAEs, Flow-based models and Diffusion Models. This practical will walk you through the challenges involved in developing an effective generative model for the particular case of Denoise Diffusion Models (a.k.a. a Score-Based Generative Model), the backbone of the recent and exciting Dalle-2 and Imagen models that we’ve all seen on Twitter. |
| Intro to Reinforcement Learning |
In Reinforcement Learning (RL), an agent is trained to take the best actions in an environment so as to maximize a given reward in the long run. RL has seen tremendous success on a wide range of challenging problems such as learning to play complex video games like Atari, StarCraft II and Dota II. In this introductory tutorial we will explore various RL approaches for solving the classic CartPole, an inverted pendulum system, where an agent must learn to balance a vertical pole by displacing the cart. |
| Social Impacts of Artificial Intelligence |
In this practical you will learn to use different tools to assess how stereotypes can result in discriminatory behavior in language technologies and reflect on their social impacts. You will explore biases in large language models and word embeddings. Working in groups, you will find strengths and limitations in the representations of stereotypes and the different approaches to model bias. This is an introductory course and no previous NLP knowledge is required. More information at this living site. |
The above practicals are derived from the practicals developed for Indaba 2022 with kind permission from the authors.