README.md

Line Trace DQN

Line trace robot simulation with reinforcement learning, Deep Q-Network.

The final report of "Intelligent Informatics", Creative Informatics, the University of Tokyo.

Requirements

• Ubuntu 18.04
• ROS Melodic
• Python 3.6.x (must be installed on the system like /usr/bin/python3)

Installation

Clone packages.

$ mkdir -p ~/catkin_ws/src  
$ cd ~/catkin_ws/src  
$ wstool init .  
$ wstool set --git line_trace_dqn [email protected]:ketaro-m/line_trace_dqn.git -y  
$ wstool update  
$ wstool merge -t . line_trace_dqn/.rosinstall  
$ wstool update  

Build packages.

$ cd ..  
$ rosdep install -y -r --from-paths src --ignore-src  
$ source /opt/ros/${ROS_DISTRO}/setup.bash  
$ catkin build line_trace_dqn  
$ source ./devel/setup.bash

Usage

1. Launch Gazebo simulator

$ roslaunch line_trace_dqn turtlebot3_linetrace.launch  

(ex. Python sample scripts)

$ roslaunch line_trace_dqn test_rospy.launch script:=follower  

2. Train DQN

$ roslaunch line_trace_dqn result_graph.launch # realtime score/action plot  
$ # roslaunch line_trace_dqn turtlebot3_dqn_train.launch --ros-args # see the parameter descriptions  
$ roslaunch line_trace_dqn turtlebot3_dqn_train.launch lr:=0.1  

3. Plot learning results

$ rosrun line_trace_dqn log_plotter.py  

References

Gazebo setup

• ROSの勉強　第24弾：ライントレース（OpenCV）
• ROS講座37 gazebo worldを作成する
  • How to include uri relatively?
  • Gazebo UR [Err] [REST.cc:205] Error in REST request

Gazebo Tips

• How to reset the simulation ?

Python3

• ROSでPython3を使う方法
• Python3のvirtualenvでrospyを使う
• catkin_virtualenv
• Installing Python scripts and modules

Sample

• ROSの勉強　第12弾：センシングと移動
• PythonでTurtlebot3を動かしてみた(gazeboを使って)

OpenCV

• ROSの勉強　第45弾：ROS (melodic)においてpython3系でcv_bridgeが使えないことへの解決

DQN

• DQN（Deep Q Network）を理解したので、Gopherくんの図を使って説明
• ゼロからDeepまで学ぶ強化学習
• Pytorch, REINFORCEMENT LEARNING (DQN) TUTORIAL
• ROBOTIS e-Manual, 9. Machine Learning
• ライントレーサーをDeep Q Learningで教育する - Chainer
• 自前の環境で深層強化学習