Learning in Multiagent Systems by G. Weiß and S. Sen. Springer, 1996.
Learning and intelligence are intimately related to each other. It is usually agreed that a system capable of learning deserves to be called intelligent; and conversely, a system being considered as intelligent is, among other things, usually expected to be able to learn. Learning always has to do with the self-improvement of future behavior based on past experience. More precisely, according to the standard artificial intelligence (AI) point of view learning can be informally defined as follows:-
Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence by Gerhard Weiss. MIT Press 2000
This is the first comprehensive introduction to multiagent systems and contemporary distributed artificial intelligence that is suitable as a textbook. The book provides detailed coverage of basic topics as well as several closely related ones.Unlike traditional textbooks, the book brings together many leading experts, guaranteeing a broad and diverse base of knowledge and expertise. It emphasizes aspects of both theory and application, and provides many illustrations and examples. Also included are thought-provoking exercises of varying degrees of difficulty and a twenty-page glossary of terms found in the study of agents, multiagent systems, and distributed artificial intelligence.The book can be used for teaching as well as self-study, and is designed to meet the needs of both researchers and practitioners. In view of the interdisciplinary nature of the field, it will be a useful reference not only for computer scientists and engineers, but for social scientists and management and organization scientists as well.-
Multiagent Systems: A Survey from a Machine Learning Perspective by Peter Stone, Manuela Veloso. Autonomous Robots, 2000
Distributed Artificial Intelligence (DAI) has existed as a subfield of AI for less than two decades. DAI is concerned with systems that consist of multiple independent entities that interact in a domain. Traditionally, DAI has been divided into two sub-disciplines: Distributed Problem Solving (DPS) focuses on the information management aspects of systems with several components working together towards a common goal; Multiagent Systems (MAS) deals with behavior management in collections of several independent entities, or agents. This survey of MAS is intended to serve as an introduction to the field and as an organizational framework. A series of general multiagent scenarios are presented. For each scenario, the issues that arise are described along with a sampling of the techniques that exist to deal with them. The presented techniques are not exhaustive, but they highlight how multiagent systems can be and have been used to build complex systems. When options exist, the techniques presented are biased towards machine learning approaches. Additional opportunities for applying machine learning to MAS are highlighted and robotic soccer is presented as an appropriate test bed for MAS. This survey does not focus exclusively on robotic systems. However, we believe that much of the prior research in non-robotic MAS is relevant to robotic MAS, and we explicitly discuss several robotic MAS, including all of those presented in this issue.-
Multi-Agent Reinforcement Learning: a critical survey Yoav Shoham & Rob Powers & Trond Grenager. Stanford University 2003
We survey the recent work in AI onmulti-agent reinforcement learning(that is, learning in stochastic games). We then argue that, while exciting,this work is flawed. The fundamental flaw is unclarity about the problemor problems being addressed. After tracing a representative sample of therecent literature, we identify four well-defined problems in multi-agent reinforcement learning, single out the problem that in our view is most suitable for AI, and make some remarks about how we believe progress is to be made on this problem-
On the agenda(s) of research on multi-agent learning by Y. Shoham, R. Powers, and T. Grenager. AAAI Fall Symposium, 2004.
We survey the recent work in AI on multi-agent reinforcement learning (that is, learning in stochastic games). After tracing a representative sample of the recent literature, we argue that, while exciting, much of this work suffers from a fundamental lack of clarity about the problem or problems being addressed. We then propose five well-defined problems in multi-agent reinforcement learning and single out one that in our view is both well-suited for AI and has not yet been adequately addressed. We conclude with some remarks about how we believe progress is to be made on this problem.-
Multiagent Reinforcement Learning for Multi-Robot Systems: A Survey by Erfu Yang, Dongbing Gu. Technical report, 2004
Multiagent reinforcement learning for multirobot systems is a challenging issue in both robotics and artificial intelligence. With the ever increasing interests in theoretical researches and practical applications, currently there have been a lot of efforts towards providing some solutions to this challenge. However, there are still many difficulties in scaling up the multiagent reinforcement learning to multi-robot systems. The main objective of this paper is to provide a survey, though not completely on the multiagent reinforcement learning in multi-robot systems. After reviewing important advances in this field, some challenging problems and promising research directions are analyzed. A concluding remark is made from the perspectives of the authors.-
Evolutionary game theory and multi-agent reinforcement learning by Karl Tuyls and Ann Nowe. Cambridge University Press 2005
In this paper we survey the basics of reinforcement learning and (evolutionary) game theory, applied to the field of multi-agent systems. This paper contains three parts. We start with an overview on the fundamentals of reinforcement learning. Next we summarize the most important aspects of evolutionary game theory. Finally, we discuss the state-of-the-art of multi-agent reinforcement learning and the mathematical connection with evolutionary game theory.-
Cooperative Multi-Agent Learning: The State of the Art y Liviu Panait and Sean Luke. JAAMAS 2005
Cooperative multi-agent systems (MAS) are ones in which several agents attempt, through their interaction, to jointly solve tasks or to maximize utility. Due to the interactions among the agents, multi-agent problem complexity can rise rapidly with the number of agents or their behavioral sophistication. The challenge this presents to the task of programming solutions to MAS problems has spawned increasing interest in machine learning techniques to automate the search and optimization process. We provide a broad survey of the cooperative multi-agent learning literature. Previous surveys of this area have largely focused on issues common to specific subareas (for example, reinforcement learning, RL or robotics). In this survey we attempt to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics. We find that this broad view leads to a division of the work into two categories, each with its own special issues: applying a single learner to discover joint solutions to multi-agent problems (team learning), or using multiple simultaneous learners, often one per agent (concurrent learning). Additionally, we discuss direct and indirect communication in connection with learning, plus open issues in task decomposition, scalability, and adaptive dynamics. We conclude with a presentation of multi-agent learning problem domains, and a list of multi-agent learning resources. Keywords: multi-agent systems, machine learning, multi-agent learning, cooperation, survey.-
An overview of cooperative and competitive multiagent learning by Pieter Jan 't Hoen, Karl Tuyls, Liviu Panait, Sean Luke and J. A. La Poutré. AAMAS 2005
Multi-agent systems (MASs) is an area ofdistributed artifi-cial intelligence that emphasizes the joint behaviors of agents with somedegree of autonomy and the complexities arising from their interactions.The research on MASs is intensifying, as supported by a growing num-ber of conferences, workshops, and journal papers. In this survey we givean overview of multi-agent learning research in a spectrum of areas, in-cluding reinforcement learning, evolutionary computation, game theory,complex systems, agent modeling, and robotics.MASs range in their description from cooperative to being competitivein nature. To muddle the waters, competitive systems can show appar-ent cooperative behavior, and vice versa. In practice, agents can showa wide range of behaviors in a system, that may either fit the label ofcooperative or competitive, depending on the circumstances. In this sur-vey, we discuss current work on cooperative and competitive MASs andaim to make the distinctions and overlap between the two approachesmore explicit.Lastly, this paper summarizes the papers of the first International work-shop on Learning and Adaptation in MAS (LAMAS) hosted at the fourthInternational Joint Conference on Autonomous Agents and Multi AgentSystems (AAMAS’05) and places the work in the above survey.-
If multi-agent learning is the answer, what is the question? by Yoav Shoham, Rob Powers and Trond Grenager. Elsevier 2007
The area of learning in multi-agent systems is today one of the most fertile grounds for interaction between game theory and artificial intelligence. We focus on the foundational questions in this interdisciplinary area, and identify several distinct agendas that ought to, we argue, be separated. The goal of this article is to start a discussion in the research community that will result in firmer foundations for the area.-
Multiagent learning is not the answer. It is the question by Peter Stone. Elsevier 2007
The article by Shoham, Powers, and Grenager called “If multi-agent learning is the answer, what is the question?” does a great job of laying out the current state of the art and open issues at the intersection of game theory and artificial intelligence (AI). However, from the AI perspective, the term “multiagent learning” applies more broadly than can be usefully framed in game theoretic terms. In this larger context, how (and perhaps whether) multiagent learning can be usefully applied in complex domains is still a large open question.-
A comprehensive survey of multi-agent reinforcement learning by Lucian Buşoniu, Robert Babuška and Bart De Schutte. IEEE Transactions on Systems, Man, and Cybernetics, 2008
Multi-agent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must instead discover a solution on their own, using learning. A significant part of the research on multi-agent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multi-agent reinforcement learning (MARL). A central issue in the field is the formal statement of the multi-agent learning goal. Different viewpoints on this issue have led to the proposal of many different goals, among which two focal points can be distinguished: stability of the agents’ learning dynamics, and adaptation to the changing behavior of the other agents. The MARL algorithms described in the literature aim—either explicitly or implicitly—at one of these two goals or at a combination of both, in a fully cooperative, fully competitive, or more general setting. A representative selection of these algorithms is discussed in detail in this paper, together with the specific issues that arise in each category. Additionally, the benefits and challenges of MARL are described along with some of the problem domains where MARL techniques have been applied. Finally, an outlook for the field is provided.-
Multiagent systems: Algorithmic, game-theoretic, and logical foundations by Yoav Shoham and Kevin Leyton-Brown. Cambridge University Press 2009
TODO: This is a book, we have to make our own abstract-
An Introduction to MultiAgent Systems by Michael Wooldridge. John Wiley & Sons 2009
Multiagent systems are a new paradigm for understanding and building distributed systems, where it is assumed that the computational components are autonomous: able to control their own behaviour in the furtherance of their own goals. The first edition of An Introduction to Multiagent Systems was the first contemporary textbook in the area, and became the standard undergraduate reference work for the field. This second edition has been extended with substantial new material on recent developments in the field, and has been revised and updated throughout. It provides a comprehensive, coherent, and readable introduction to the theory and practice of multiagent systems, while presenting a wealth of discussion topics and pointers into more advanced issues for those wanting to dig deeper.-
Multi-agent reinforcement learning:An overview by Lucian Buşoniu, Robert Babuška and Bart De Schutte. Springer 2010
Multi-agent systems can be used to address problems in a variety of do-mains, including robotics, distributed control, telecommunications, andeconomics.The complexity of many tasks arising in these domains makes them difficult tosolvewith preprogrammed agent behaviors. The agents must instead discover a solutionon their own, using learning. A significant part of the research on multi-agentlearn-ing concerns reinforcement learning techniques. This chapter reviews arepresenta-tive selection of Multi-Agent Reinforcement Learning (MARL) algorithms for fullycooperative, fully competitive, and more general (neither cooperative norcompeti-tive) tasks. The benefits and challenges of MARL are described. A centralchallengein the field is the formal statement of a multi-agent learning goal; thischapter re-views the learning goals proposed in the literature. The problem domains whereMARL techniques have been applied are briefly discussed. Several MARL algo-rithms are applied to an illustrative example involving the coordinatedtransporta-tion of an object by two cooperative robots. In an outlook for the MARL field,a setof important open issues are identified, and promising research directions toaddressthese issues are outlined.-
Multiagent Learning: Basics, Challenges, and Prospects by Karl Tuyls and Gerhard Weiss. AI magazine, 2012 [Summary]
Multiagent systems (MAS) are widely accepted as an important method for solving problems of a distributed nature. A key to the success of MAS is efficient and effective multiagent learning (MAL). The past 25 years have seen a great interest and tremendous progress in the field of MAL. This article introduces and overviews this field by presenting its fundamentals, sketching its historical development, and describing some key algorithms for MAL. Moreover, main challenges that the field is facing today are identified.-
Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems by Laetitia Matignon, Guillaume J. Laurent and Nadine Le Fort-Piat. Cambridge University Press 2012
In the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate. This paper identifies several challenges responsible for the non-coordination of independent agents: Pareto-selection non-stationarity, stochasticity, alter-exploration and shadowed equilibria. A selection of multi-agent domains is classified according to those challenges: matrix games,Boutilier’s coordination game, predators pursuit domains and a special multi-state game.Moreover, the performance of a range of algorithms for independent reinforcementlearners is evaluated empirically.Those algorithms are Q-learning variants: decentralized Q-learning, distributedQ-learning, hystereticQ-learning, recursive frequency maximum Q-value and win-or-learn fast policy hillclimbing. Anoverview of the learning algorithms’ strengths and weaknesses against each challengeconcludes thepaper and can serve as a basis for choosing the appropriate algorithm for a new domain.Furthermore,the distilled challenges may assist in the design of new learning algorithms thatovercome theseproblems and achieve higher performance in multi-agent applications.-