Intro to Mathematical Optimization

1) Intro (10 minutes)

The Traveling Saleman Problem
What to expect in this course
Machine learning vs optimization

2) Metaheuristics (30 minutes)

Traveling Salesman Problem explained
Linear regression with hill climbing/simulated annealing
Linear quantile regression with hill climbing/simulated annealing
Other metaheuristics algorithms
EXERCISE: Multiple choice, which hyperparameters need to be fixed?
- Too greedy/random temperature schedule
- Not enough iterations
- Not enough move variety

BREAK (10 minutes)

3) Tree Search Algorithms (30 minutes)

Solving a Sudoku
Creating a schedule under constraints
Knapsack Problem/Branch-and-bound Intuition
Mixing search algorithms with simulations
EXERCISE: Choosing the right heuristics
- Knapsack: which items should I grab first?
- Knapsack: what's the relaxation value?
- Scheduling: which employees should I schedule first?
- Scheduling: upon scheduling this employee, what can I prune?

BREAK (10 minutes)

4) Linear/Integer/Mixed Programming (30 minutes)

Basic Convex Optimization Intuition
Linear Programming
Blending Problem
Manufacturing problem
Continuous Scheduling Problem
Using libraries for linear/integer/mixed programming
- Python - PyOmo/PuLp
- Java - ojAlgo
EXERCISES
- Convex/Not Convex?
- Choosing the right objective expression
- Choosing the right constraints

BREAK (10 minutes)

5) Gradient Descent (30 minutes)

Understanding Gradient Descent
Doing linear regression with gradient descent
Partial derivatives
How neural networks use gradient descent
Types of gradient descent
EXERCISE
- Learning rate too big/small
- Machine learning versus optimization - which to use?
POST-CLASS HOMEWORK