02_ml_models

Using a Machine Learning Model

Session A: Introduction to ml5.js and pre-trained models

Objectives:

• Understand the concept of a “machine learning model.”
• What is a “pre-trained model”?
• What does it mean to discuss the “architecture” of a machine learning model?
• Define and diagram an artificial neural network.
• Understand what ml5.js is and how it fits into the TensorFlow and open source machine learning library ecosystem.
• Learn how to create an image classifier with ml5.js and MobileNet.
• Understand how the MobileNet model was trained, specifically the origins and collection methodology for the training.
• Week 2 Slides

ml5.js video tutorials

• A Beginner's Guide to Machine Learning with ml5.js - video tutorial
• ml5.js: Image Classification with MobileNet - video tutorial
• ml5.js: Webcam Image Classification - video tutorial

ml5.js resources

• ml5 website
• ml5 web editor examples

ml5 code editor examples

• Image Classification
• Drag and Drop Image Classification
• Webcam Image Classification
• Webcam Image Classification Text to Speech
• Image Classification Multiple Images
• Image Classification Scavenger Hunt

Defining Machine Learning yet again

• "Machine learning is programming with examples, not instructions" -- from Kyle McDonald - Weird Intelligence.
• "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E." -- Tom Mitchell (1998): Machine Learning book.
  • Example: classifying images of dogs and cats.
    • E = Watching you classify images as dogs or cats.
    • T = Classifying images as dogs or cats.
    • P = The % of images correctly classified.

Classification and Regression

• Classification and regression both involve making a "prediction" based on input data.
• Classification refers to predicting an output with a discrete set of possibilities like a set of categories or labels. For example: "Given an input image, is it a dog or cat?"
• Regression refers to predicting an "continuous" output (a fancy way of saying number). For example: "Given the number of bedrooms, what is the price of a house?" or "Given an input image of a cat, how much does the cat weigh?"

Reading / Viewing:

1. Read Andrey Kurenkov's 'Brief' History of Neural Nets and Deep Learning
2. Read ImageNet: The Data That Transformed AI Research—and Possibly the World by Dave Gershgorn (Note: Fei-Fei Li is no longer at Google; she is currently Co-Director of the Stanford Human-Centered AI Institute)

Assignment 2A

1. Explore ImageNet. What surprises you about this data set? What questions do you have? Thinking back to last week’s assignment, can you think of any ethical considerations around how this data was collected Are there privacy considerations with the data?
2. Using the ml5.js examples above, try running image classification on a variety of images. Pick at least 10 objects in your room. How many of these does it recognize? What other aspects of the image affect the classification, including but not limited to position, scale, lighting, etc.
3. Document your thoughts on MobileNet and image classification in a blog post and add a link to the Assignment 2A Wiki.

Session B: Transfer learning

Objectives:

• Diagram and define the layers of a neural network, introducing the concept of a “convolutional layer.”
• Distinguish between a “feature vector” (aka logits) and the last layer (aka softmax probabilities) of a classification network.
• Understand the process of “transfer learning.”
• Learn to save a trained model for re-use later.
• Week 2 Slides

Supervised Learning

• From Andrew Ng: "In supervised learning, we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output."
• Adapted from Nature of Code Chapter 10: Supervised Learning is a strategy that involves a "teacher" that trains the learning system. For example, consider facial recognition. The "teacher" shows the network a bunch of faces (the teacher already knows the names associated with each face). The learning system makes its guesses and the teacher provides the answers. The learning system can then compare its answers to the known “correct” ones and make adjustments according to its errors.

ml5.js video tutorials

• ml5.js: Transfer Learning with Feature Extractor - video tutorial
• ml5.js: Feature Extractor Classification - video tutorial
• ml5.js: Feature Extractor Regression - video tutorial
• ml5.js: Save/Load Model - video tutorial

Related Projects:

• Webcam Pacman by Google’s TensorFlow.js Team
• Pong ML by Alejandro Matamala Ortiz
• Regression Curve by Joohyun Park
• Asemic Writing Teachable Machine by Jillian Zhong
• Resistor classification by Azalea Vaseghi and Amitabh Shrivastava

Reading / Viewing:

1. Watch But what is a Neural Network? by 3Blue1Brown
2. Read How to Build a Teachable Machine with TensorFlow.js
3. Read Excavating AI: The Politics of Images in Machine Learning Training Sets by Kate Crawford and Trevor Paglen (See: ImageNet Roulette - going offline Sept 27, 2019).

Assignment 2B:

1. Train your own image classifer using transfer learning and ml5.js and apply the model to an interactive p5.js sketch. You can train the model with Teachable Machine (see links provided over e-mail) or with your own ml5.js code. Feel free to try sound instead of or in addition to images. You may also choose to experiment with a "regression" rather than classification.
   • Teachable Machine ml5.js example for Image Classification
   • Teachable Machine ml5.js example for Sound Classification
2. Document your exercise in a blog post and add a link to the post and your p5 sketch on the Assignment 2B Wiki. In your blog post, include visual documentation such as a recorded screen capture / video of your training session and sketch running in the browser.