Awesome Monte Carlo Tree Search Papers.

A curated list of Monte Carlo tree search papers with implementations from the following conferences/journals:

• Machine learning
  • NeurIPS
  • ICML
• Computer vision
  • CVPR
  • ICCV
• Natural language processing
  • ACL
• Data
  • KDD
• Artificial intelligence
  • AAAI
  • AISTATS
  • IJCAI
  • UAI
• Robotics
  • RAS

Similar collections about graph classification, gradient boosting, classification/regression trees, fraud detection, and community detection papers with implementations.

2019

• ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search (AAAI 2019)

  • Shangtong Zhang, Hengshuai Yao
  • [Paper]
  • [Code]

• A Monte Carlo Tree Search Player for Birds of a Feather Solitaire (AAAI 2019)

  • Christian Roberson, Katarina Sperduto
  • [Paper]
  • [Code]

• Vine Copula Structure Learning via Monte Carlo Tree Search (AISTATS 2019)

  • Bo Chang, Shenyi Pan, Harry Joe
  • [Paper]
  • [Code]

• Noisy Blackbox Optimization using Multi-fidelity Queries: A Tree Search Approach (AISTATS 2019)

  • Rajat Sen, Kirthevasan Kandasamy, Sanjay Shakkottai
  • [Paper]
  • [Code]

• Monte Carlo Tree Search for Policy Optimization (IJCAI 2019)

  • Xiaobai Ma, Katherine Rose Driggs-Campbell, Zongzhang Zhang, Mykel J. Kochenderfer
  • [Paper]

• Subgoal-Based Temporal Abstraction in Monte-Carlo Tree Search (IJCAI 2019)

  • Thomas Gabor, Jan Peter, Thomy Phan, Christian Meyer, Claudia Linnhoff-Popien
  • [Paper]
  • [Code]

• Automated Machine Learning with Monte-Carlo Tree Search (IJCAI 2019)

  • Herilalaina Rakotoarison, Marc Schoenauer, Michèle Sebag
  • [Paper]
  • [Code]

• Multiple Policy Value Monte Carlo Tree Search (IJCAI 2019)

  • Li-Cheng Lan, Wei Li, Ting-Han Wei, I-Chen Wu
  • [Paper]

2018

• Reset-free Trial-and-Error Learning for Robot Damage Recovery (RAS 2018)

  • Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Jean-Baptiste Mouret
  • [Paper]
  • [Code]
  • [MCTS C++ Library]

• Memory-Augmented Monte Carlo Tree Search (AAAI 2018)

  • Chenjun Xiao, Jincheng Mei, Martin Müller
  • [Paper]

• Feedback-Based Tree Search for Reinforcement Learning (ICML 2018)

  • Daniel R. Jiang, Emmanuel Ekwedike, Han Liu
  • [Paper]

• Extended Increasing Cost Tree Search for Non-Unit Cost Domains (IJCAI 2018)

  • Thayne T. Walker, Nathan R. Sturtevant, Ariel Felner
  • [Paper]

• Three-Head Neural Network Architecture for Monte Carlo Tree Search (IJCAI 2018)

  • Chao Gao, Martin Müller, Ryan Hayward
  • [Paper]

• Bidding in Periodic Double Auctions Using Heuristics and Dynamic Monte Carlo Tree Search (IJCAI 2018)

  • Moinul Morshed Porag Chowdhury, Christopher Kiekintveld, Son Tran, William Yeoh
  • [Paper]

• Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search (NIPS 2018)

  • Zhuwen Li, Qifeng Chen, Vladlen Koltun
  • [Paper]

• M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search (NIPS 2018)

  • Yelong Shen, Jianshu Chen, Po-Sen Huang, Yuqing Guo, Jianfeng Gao
  • [Paper]

• Single-Agent Policy Tree Search With Guarantees (NIPS 2018)

  • Laurent Orseau, Levi Lelis, Tor Lattimore, Theophane Weber
  • [Paper]

• Monte-Carlo Tree Search for Constrained POMDPs (NIPS 2018)

  • Jongmin Lee, Geon-hyeong Kim, Pascal Poupart, Kee-Eung Kim
  • [Paper]

2017

• An Analysis of Monte Carlo Tree Search (AAAI 2017)

  • Steven James, George Dimitri Konidaris, Benjamin Rosman
  • [Paper]

• Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation (AAAI 2017)

  • Jinzhuo Wang, Wenmin Wang, Ronggang Wang, Wen Gao
  • [Paper]

• Designing Better Playlists with Monte Carlo Tree Search (AAAI 2017)

  • Elad Liebman, Piyush Khandelwal, Maytal Saar-Tsechansky, Peter Stone
  • [Paper]

• Learning in POMDPs with Monte Carlo Tree Search (ICML 2017)

  • Sammie Katt, Frans A. Oliehoek, Christopher Amato
  • [Paper]

• Learning to Run Heuristics in Tree Search (IJCAI 2017)

  • Elias B. Khalil, Bistra Dilkina, George L. Nemhauser, Shabbir Ahmed, Yufen Shao
  • [Paper]

• Estimating the Size of Search Trees by Sampling with Domain Knowledge (IJCAI 2017)

  • Gleb Belov, Samuel Esler, Dylan Fernando, Pierre Le Bodic, George L. Nemhauser
  • [Paper]

• A Monte Carlo Tree Search Approach to Active Malware Analysis (IJCAI 2017)

  • Riccardo Sartea, Alessandro Farinelli
  • [Paper]

• Monte-Carlo Tree Search by Best Arm Identification (NIPS 2017)

  • Emilie Kaufmann, Wouter M. Koolen
  • [Paper]

• Thinking Fast and Slow with Deep Learning and Tree Search (NIPS 2017)

  • Thomas Anthony, Zheng Tian, David Barber
  • [Paper]

• Monte-Carlo Tree Search using Batch Value of Perfect Information (UAI 2017)

  • Shahaf S. Shperberg, Solomon Eyal Shimony, Ariel Felner
  • [Paper]

2016

• Using Domain Knowledge to Improve Monte-Carlo Tree Search Performance in Parameterized Poker Squares (AAAI 2016)

  • Robert Arrington, Clay Langley, Steven Bogaerts
  • [Paper]

• Monte Carlo Tree Search for Multi-Robot Task Allocation (AAAI 2016)

  • Bilal Kartal, Ernesto Nunes, Julio Godoy, Maria L. Gini
  • [Paper]

• Large Scale Hard Sample Mining with Monte Carlo Tree Search (CVPR 2016)

  • Olivier Canévet, François Fleuret
  • [Paper]

• On the Analysis of Complex Backup Strategies in Monte Carlo Tree Search (ICML 2016)

  • Piyush Khandelwal, Elad Liebman, Scott Niekum, Peter Stone
  • [Paper]

• Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Games (IJCAI 2016)

  • Xiaoxiao Guo, Satinder P. Singh, Richard L. Lewis, Honglak Lee
  • [Paper]

• Monte Carlo Tree Search in Continuous Action Spaces with Execution Uncertainty (IJCAI 2016)

  • Timothy Yee, Viliam Lisý, Michael H. Bowling
  • [Paper]

• Learning Predictive State Representations via Monte-Carlo Tree Search (IJCAI 2016)

  • Yunlong Liu, Hexing Zhu, Yifeng Zeng, Zongxiong Dai
  • [Paper]

2015

• Efficient Globally Optimal Consensus Maximisation with Tree Search (CVPR 2015)

  • Tat-Jun Chin, Pulak Purkait, Anders P. Eriksson, David Suter
  • [Paper]

• Interplanetary Trajectory Planning with Monte Carlo Tree Search (IJCAI 2015)

  • Daniel Hennes, Dario Izzo
  • [Paper]

2014

• State Aggregation in Monte Carlo Tree Search (AAAI 2014)

  • Jesse Hostetler, Alan Fern, Tom Dietterich
  • [Paper]

• Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Tree Search Planning (NIPS 2014)

  • Xiaoxiao Guo, Satinder P. Singh, Honglak Lee, Richard L. Lewis, Xiaoshi Wang
  • [Paper]

• Learning Partial Policies to Speedup MDP Tree Search (UAI 2014)

  • Jervis Pinto, Alan Fern
  • [Paper]

2013

• Monte Carlo Tree Search for Scheduling Activity Recognition (ICCV 2013)

  • Mohamed R. Amer, Sinisa Todorovic, Alan Fern, Song-Chun Zhu
  • [Paper]

• Convergence of Monte Carlo Tree Search in Simultaneous Move Games (NIPS 2013)

  • Viliam Lisý, Vojtech Kovarík, Marc Lanctot, Branislav Bosanský
  • [Paper]

• Bayesian Mixture Modelling and Inference based Thompson Sampling in Monte-Carlo Tree Search (NIPS 2013)

  • Aijun Bai, Feng Wu, Xiaoping Chen
  • [Paper]

2012

• Generalized Monte-Carlo Tree Search Extensions for General Game Playing (AAAI 2012)
  • Hilmar Finnsson
  • [Paper]

2011

• A Local Monte Carlo Tree Search Approach in Deterministic Planning (AAAI 2011)

  • Fan Xie, Hootan Nakhost, Martin Müller
  • [Paper]

• Real-Time Solving of Quantified CSPs Based on Monte-Carlo Game Tree Search (IJCAI 2011)

  • Satomi Baba, Yongjoon Joe, Atsushi Iwasaki, Makoto Yokoo
  • [Paper]

• Nested Rollout Policy Adaptation for Monte Carlo Tree Search (IJCAI 2011)

  • Christopher D. Rosin
  • [Paper]

• Variance Reduction in Monte-Carlo Tree Search (NIPS 2011)

  • Joel Veness, Marc Lanctot, Michael H. Bowling
  • [Paper]

• Learning Is Planning: Near Bayes-Optimal Reinforcement Learning via Monte-Carlo Tree Search (UAI 2011)

  • John Asmuth, Michael L. Littman
  • [Paper]

2010

• Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search (AAAI 2010)

  • Jeffrey Richard Long, Nathan R. Sturtevant, Michael Buro, Timothy Furtak
  • [Paper]

• Bayesian Inference in Monte-Carlo Tree Search (UAI 2010)

  • Gerald Tesauro, V. T. Rajan, Richard Segal
  • [Paper]

2009

• Monte Carlo Tree Search Techniques in the Game of Kriegspiel (IJCAI 2009)

  • Paolo Ciancarini, Gian Piero Favini
  • [Paper]

• Bootstrapping from Game Tree Search (NIPS 2009)

  • Joel Veness, David Silver, William T. B. Uther, Alan Blair
  • [Paper]

2008

• Direct Mining of Discriminative and Essential Frequent Patterns via Model-Based Search Tree (KDD 2008)
  • Wei Fan, Kun Zhang, Hong Cheng, Jing Gao, Xifeng Yan, Jiawei Han, Philip S. Yu, Olivier Verscheure
  • [Paper]

2007

• Bandit Algorithms for Tree Search (UAI 2007)
  • Pierre-Arnaud Coquelin, Rémi Munos
  • [Paper]

2006

• Properties of Forward Pruning in Game-Tree Search (AAAI 2006)

  • Yew Jin Lim, Wee Sun Lee
  • [Paper]

• Graph Branch Algorithm: An Optimum Tree Search Method for Scored Dependency Graph with Arc Co-Occurrence Constraints (ACL 2006)

  • Hideki Hirakawa
  • [Paper]

2005

• Game-Tree Search with Combinatorially Large Belief States (IJCAI 2005)
  • Austin Parker, Dana S. Nau, V. S. Subrahmanian
  • [Paper]

2003

• Solving Finite Domain Constraint Hierarchies by Local Consistency and Tree Search (IJCAI 2003)
  • Stefano Bistarelli, Philippe Codognet, Kin Chuen Hui, Jimmy Ho-Man Lee
  • [Paper]

2001

• Incomplete Tree Search using Adaptive Probing (IJCAI 2001)
  • Wheeler Ruml
  • [Paper]

1998

• KnightCap: A Chess Programm That Learns by Combining TD with Game-Tree Search (ICML 1998)
  • Jonathan Baxter, Andrew Tridgell, Lex Weaver
  • [Paper]

1988

• A Tree Search Algorithm for Target Detection in Image Sequences (CVPR 1988)
  • Steven D. Blostein, Thomas S. Huang
  • [Paper]