Skip to content

Files

Latest commit

 

History

History

week3_model_free

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 

Lecture slides - here

Materials

  • Russian materials:
  • English materials:
    • Lecture by David Silver (english) - video part I, video part II
    • Alternative lecture by Pieter Abbeel (english) - video
    • Alternative lecture by John Schulmann (english) - video
    • Blog post on q-learning Vs SARSA - url

More materials

  • N-step temporal difference from Sutton's book - suttonbook chapter 7
  • Eligibility traces from Sutton's book - suttonbook chapter 12
  • Blog post on eligibility traces - url

Assignments

This week's practice will require you to pick one of ./seminar_main and ./seminar_alternative as first part.

Then ./homework and follow instructions in ./homework/homework.ipynb

Below are some guidelines on what to do in seminar_main/_alternative.

./seminar_main

this assignment borrows code from awesome cs188 This homework assignment works on python2 only. If you stick to py3, consider seminar_alternative. Or just install it for this homework alone and remove afterwards.

This homework also requires some physical display (e.g. laptop monitor). It won't work on binder VM / headless server. Please run it on laptop or consider ./seminar_alternative

  • You need to implement QLearining algorithm. If you're running go to seminar_main/ folder and open file qlearningAgent.py.

Once you're done, run use those commands:

python crawler.py # Crawler with qlearning
python pacman.py -p <your agent> -x <number of train samples> -n <total number of samples> -l <grid env>
python pacman.py -p PacmanQAgent -x 5000 -n 5010 -l smallGrid # example
  • Make sure you can tune agent to beat ./run_crawler.sh
  • on windows, just run python crawler.py from cmd in the project directory
  • other ./run* files are mostly for your amusement.
    • ./run_pacman.sh will need more epochs to converge, see comments
    • on windows, just copy the type python pacman.py -p PacmanQAgent -x 2000 -n 2010 -l smallGrid in cmd from assignemnt dir (YSDA/HSE) Please submit only qlearningAgents.py file and include a brief text report as comments in it.

./seminar_alternative

You'll have to implement qlearning.py just like in main seminar, but in ./seminar_alternative folder. After you're done with it, open the seminar notebook and follow instructions from there.