diff --git a/README.md b/README.md index 403ec1dc..c8d0031d 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,8 @@ The [D4RL](https://github.com/Farama-Foundation/D4RL) environments are now avail The different tasks involve hammering a nail, opening a door, twirling a pen, or picking up and moving a ball. * [Franka Kitchen](https://robotics.farama.org/envs/franka_kitchen/) - Multitask environment in which a 9-DoF Franka robot is placed in a kitchen containing several common household items. The goal of each task is to interact with the items in order to reach a desired goal configuration. +* [MaMuJoCo](https://robotics.farama.org/envs/MaMuJoCo/) - A collection of multi agent factorizations of the [Gymnasium/MuJoCo](https://gymnasium.farama.org/environments/mujoco/) environments and a framework for factorizing robotic environments, uses the [pettingzoo.ParallelEnv](https://pettingzoo.farama.org/api/parallel/) API. + **WIP**: generate new `D4RL` environment datasets with [Minari](https://github.com/Farama-Foundation/Minari). ## Multi-goal API diff --git a/docs/envs/MaMuJoCo/index.md b/docs/envs/MaMuJoCo/index.md index b70a16a5..94d2b9b0 100644 --- a/docs/envs/MaMuJoCo/index.md +++ b/docs/envs/MaMuJoCo/index.md @@ -19,20 +19,19 @@ Gymansium-Robotics/MaMuJoCo Represents the first, easy to use Framework for rese MaMuJoCo mainly uses the [PettingZoo.ParallelAPI](https://pettingzoo.farama.org/api/parallel/), but also supports a few extra functions: ```{eval-rst} -.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_local_actions_to_global_action +.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_local_actions_to_global_action ``` - ```{eval-rst} -.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_global_action_to_local_actions +.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_global_action_to_local_actions ``` ```{eval-rst} -.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_global_state_to_local_observations +.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_global_state_to_local_observations ``` ```{eval-rst} -.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_local_observation_to_global_state +.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_local_observations_to_global_state ``` ```{eval-rst} -.. autofunction:: gymnasium_robotics.mamujoco_v0.get_parts_and_edges +.. autofunction:: gymnasium_robotics.mamujoco_v1.get_parts_and_edges ``` MaMuJoCo also supports the [PettingZoo.AECAPI](https://pettingzoo.farama.org/api/aec/) but does not expose extra functions. @@ -41,19 +40,19 @@ MaMuJoCo also supports the [PettingZoo.AECAPI](https://pettingzoo.farama.org/api ### Arguments ```{eval-rst} -.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.__init__ +.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.__init__ ``` ## How to create new agent factorizations -### example 'Ant-v4', '8x1' +### example 'Ant-v5', '8x1' In this example, we will create an agent factorization not present in Gymnasium-Robotics/MaMuJoCo the "Ant"/'8x1', where each agent controls a single joint/action (first implemented by [safe-MaMuJoCo](https://github.com/chauncygu/Safe-Multi-Agent-Mujoco)). first we will load the graph of MaMuJoCo: ```python ->>> from gymnasium_robotics.mamujoco_v0 import get_parts_and_edges ->>> unpartioned_nodes, edges, global_nodes = get_parts_and_edges('Ant-v4', None) +>>> from gymnasium_robotics.mamujoco_v1 import get_parts_and_edges +>>> unpartioned_nodes, edges, global_nodes = get_parts_and_edges('Ant-v5', None) ``` The `unpartioned_nodes` contain the nodes of the MaMuJoCo graph. The `edges` well, contain the edges of the graph. @@ -63,18 +62,23 @@ To create our '8x1' partition we will need to partition the `unpartioned_nodes`: ```python >>> unpartioned_nodes [(hip1, ankle1, hip2, ankle2, hip3, ankle3, hip4, ankle4)] ->>> partioned_nodes = [(unpartioned_nodes[0][0],), (unpartioned_nodes[0][1],), (unpartioned_nodes[0][2],), (unpartioned_nodes[0][3],), (unpartioned_nodes[0][4],), (unpartioned_nodes[0][5],), (unpartioned_nodes[0][6],), (unpartioned_nodes[0][7],)]>>> partioned_nodes +>>> partioned_nodes = [(unpartioned_nodes[0][0],), (unpartioned_nodes[0][1],), (unpartioned_nodes[0][2],), (unpartioned_nodes[0][3],), (unpartioned_nodes[0][4],), (unpartioned_nodes[0][5],), (unpartioned_nodes[0][6],), (unpartioned_nodes[0][7],)] >>> partioned_nodes [(hip1,), (ankle1,), (hip2,), (ankle2,), (hip3,), (ankle3,), (hip4,), (ankle4,)] ``` Finally package the partitions and create our environment: ```python >>> my_agent_factorization = {"partition": partioned_nodes, "edges": edges, "globals": global_nodes} ->>> gym_env = mamujoco_v0('Ant', '8x1', agent_factorization=my_agent_factorization) +>>> gym_env = mamujoco_v1('Ant', '8x1', agent_factorization=my_agent_factorization) ``` ## Version History -v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of [the original multiagent_mujuco](https://github.com/schroederdewitt/multiagent_mujoco) +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). + - When `factorizatoion=None`, the `env.gent_action_partitions.dummy_node` now contains `action_id` (it used to be `None`). + - Added `map_local_observations_to_global_state` & optimized runtime performance of `map_global_state_to_local_observations`. + - Added `gym_env` argument which can be used to load third-party `Gymansium.MujocoEnv` environments. +* v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of [the original multiagent_mujuco](https://github.com/schroederdewitt/multiagent_mujoco) ```{toctree} :hidden: @@ -84,6 +88,7 @@ ma_half_cheetah.md ma_hopper.md ma_humanoid_standup.md ma_humanoid.md +ma_multiagentswimmer.md ma_reacher.md ma_swimmer.md ma_pusher.md diff --git a/docs/envs/MaMuJoCo/ma_ant.md b/docs/envs/MaMuJoCo/ma_ant.md index d1bb06f4..150de59a 100644 --- a/docs/envs/MaMuJoCo/ma_ant.md +++ b/docs/envs/MaMuJoCo/ma_ant.md @@ -15,7 +15,7 @@ This Environment is part of [MaMuJoCo](https://robotics.farama.org/envs/MaMuJoCo The task is [Gymansium's MuJoCo/Ant](https://gymnasium.farama.org/environments/mujoco/ant/). -| Defaults | `env = mamujoco_v0.parallel_env("Ant", None)` | +| Defaults | `env = mamujoco_v1.parallel_env("Ant", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -23,14 +23,14 @@ The task is [Gymansium's MuJoCo/Ant](https://gymnasium.farama.org/environments/m | Part partition | `[(hip4, ankle4, hip1, ankle1, hip2, ankle2, hip3, ankle3)]` | ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/ant.png :name: ant ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Ant", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Ant", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -56,7 +56,7 @@ If partitioning, is None then the environment contains a single agent with the s :name: ant_2x4 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Ant", "2x4")` | +| Instantiate | `env = mamujoco_v1.parallel_env("Ant", "2x4")` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -86,7 +86,7 @@ The environment is partitioned in 2 parts, the front part (containing the front :name: ant_2x4d ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Ant", "2x4d")` | +| Instantiate | `env = mamujoco_v1.parallel_env("Ant", "2x4d")` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -114,7 +114,7 @@ The environment is partitioned in 2 parts, split diagonally. :name: ant_4x2 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Ant", "4x2")` | +| Instantiate | `env = mamujoco_v1.parallel_env("Ant", "4x2")` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1', 'agent_2', 'agent_3']` | | Number of Agents | 4 | @@ -168,7 +168,7 @@ All agents receive the same [Gymnasium's Ant](https://gymnasium.farama.org/envir ## Starting state -The starting state of the environment is the as [Gymnasium's Ant](https://gymnasium.farama.org/environments/mujoco/ant/#starting-state). +The starting state of the environment is the same as [Gymnasium's Ant](https://gymnasium.farama.org/environments/mujoco/ant/#starting-state). @@ -178,7 +178,11 @@ All agent terminate and truncate at the same time given the same conditions as [ ## Version History -- v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). + - Now observes `local_categories` of `cfrc_ext` by default (same as `Gymnasium/MuJoCo-v5/Ant`). + - Renamed global node `torso` → `root`. +* v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Fixed diagonal factorization ("2x4d") not being diagonal. - Fixed Global observations (The Ant's Torso: `rootx`, `rooty`, `rootz`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_coupled_half_cheetah.md b/docs/envs/MaMuJoCo/ma_coupled_half_cheetah.md index 3acaae3e..da6d89ef 100644 --- a/docs/envs/MaMuJoCo/ma_coupled_half_cheetah.md +++ b/docs/envs/MaMuJoCo/ma_coupled_half_cheetah.md @@ -11,14 +11,14 @@ The environment consists of 2 half cheetahs coupled by an elastic tendon. ## Action Space -The action spaces is depended on the partitioning. +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/coupled_half_cheetah.png :name: coupled_half_cheetah ``` -| Instantiate | `env = mamujoco_v0.parallel_env("CoupledHalfCheetah", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("CoupledHalfCheetah", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -49,7 +49,7 @@ If partitioning, is `None`, then the environment contains a single agent with th :name: coupled_half_cheetah_1p1 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("CoupledHalfCheetah", "1p1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("CoupledHalfCheetah", "1p1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -98,7 +98,7 @@ All agents receive the same average reward of each cheetah. ## Starting state -The starting state of the environment is the as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state) (but with 2 cheetahs). +The starting state of the environment is the same as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state) (but with 2 cheetahs). @@ -108,6 +108,8 @@ All agent terminate and truncate at the same time, given the same conditions as ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Fixed action mapping of the second cheetah (It would previously not work) diff --git a/docs/envs/MaMuJoCo/ma_half_cheetah.md b/docs/envs/MaMuJoCo/ma_half_cheetah.md index 4f463b8e..88b4a364 100644 --- a/docs/envs/MaMuJoCo/ma_half_cheetah.md +++ b/docs/envs/MaMuJoCo/ma_half_cheetah.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Half Cheetah](https://gymnasium.farama.org/envir ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/half_cheetah.png :name: half_cheetah ``` -| Instantiate | `env = mamujoco_v0.parallel_env("HalfCheetah", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("HalfCheetah", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -47,7 +47,7 @@ If partitioning, is `None`, then the environment contains a single agent with th :name: half_cheetah_2x3 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("HalfCheetah", "2x3")`| +| Instantiate | `env = mamujoco_v1.parallel_env("HalfCheetah", "2x3")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -75,7 +75,7 @@ The environment is partitioned in 2 parts, the front part (containing the front :name: half_cheetah_6x1 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("HalfCheetah", "6x1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("HalfCheetah", "6x1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1', 'agent_2', 'agent_3', 'agent_4', 'agent_5']` | | Number of Agents | 6 | @@ -129,7 +129,7 @@ All agents receive the same [Gymnasium's Half Cheetah](https://gymnasium.farama. ## Starting state -The starting state of the environment is the as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state). +The starting state of the environment is the same as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state). @@ -138,6 +138,8 @@ All agent terminate and truncate at the same time, given the same conditions as ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Added/Fixed Global observations (The Cheetah's front tip: `rootx`, `rooty`, `rootz`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_hopper.md b/docs/envs/MaMuJoCo/ma_hopper.md index bc27fb69..40bbf6f0 100644 --- a/docs/envs/MaMuJoCo/ma_hopper.md +++ b/docs/envs/MaMuJoCo/ma_hopper.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Hopper](https://gymnasium.farama.org/environment ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/hopper.png :name: hopper ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Hopper", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Hopper", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -47,7 +47,7 @@ If partitioning, is `None`, then the environment contains a single agent with th :name: hopper_3x1 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Hopper", "3x1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("Hopper", "3x1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1', 'agent_2']` | | Number of Agents | 3 | @@ -89,7 +89,7 @@ All agents receive the same [Gymnasium's Hopper](https://gymnasium.farama.org/en ## Starting state -The starting state of the environment is the as [Gymnasium's Hopper](https://gymnasium.farama.org/environments/mujoco/hopper/#starting-state). +The starting state of the environment is the same as [Gymnasium's Hopper](https://gymnasium.farama.org/environments/mujoco/hopper/#starting-state). @@ -98,6 +98,8 @@ All agent terminate and truncate at same time given the same conditions as [Gymn ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Fixed Global observations (The Hopper's top: `rootx`, `rooty`, `rootz`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_humanoid.md b/docs/envs/MaMuJoCo/ma_humanoid.md index 52291ad9..47d8afa2 100644 --- a/docs/envs/MaMuJoCo/ma_humanoid.md +++ b/docs/envs/MaMuJoCo/ma_humanoid.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Humanoid](https://gymnasium.farama.org/environme ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/humanoid.png :name: humanoid ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Humanoid", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Humanoid", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -60,7 +60,7 @@ If partitioning, is `None` then the environment contains a single agent with the :name: humanoid_9|8 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Humanoid", "3x1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("Humanoid", "3x1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -114,7 +114,7 @@ All agents receive the same [Gymnasium's Humanoid](https://gymnasium.farama.org/ ## Starting state -The starting state of the environment is the as [Gymnasium's Humanoid](https://gymnasium.farama.org/environments/mujoco/humanoid/#starting-state). +The starting state of the environment is the same as [Gymnasium's Humanoid](https://gymnasium.farama.org/environments/mujoco/humanoid/#starting-state). @@ -123,6 +123,9 @@ All agent terminate and truncate at the same time, given the same conditions as ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). + - No longer observes `qfrc_actuator` of `root` & `cinert`, `cvel`, `qfrc_actuator`, `cfrc_ext` of `worldbody` (same as `Gymnasium/MuJoCo-v5/Humanoid`). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Added/Fixed Global observations (The Humanoids's torso: `rootx`, `rooty`, `rootz`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_humanoid_standup.md b/docs/envs/MaMuJoCo/ma_humanoid_standup.md index 55369e1c..d330a7f4 100644 --- a/docs/envs/MaMuJoCo/ma_humanoid_standup.md +++ b/docs/envs/MaMuJoCo/ma_humanoid_standup.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Humanoid Standup](https://gymnasium.farama.org/e ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/humanoid.png :name: humanoid ``` -| Instantiate | `env = mamujoco_v0.parallel_env("HumanoidStandup", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("HumanoidStandup", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -60,7 +60,7 @@ If partitioning, is `None` then the environment contains a single agent with the :name: humanoid_9|8 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("HumanoidStandup", "3x1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("HumanoidStandup", "3x1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -114,7 +114,7 @@ All agents receive the same [Gymnasium's Humanoid Standup](https://gymnasium.far ## Starting state -The starting state of the environment is the as [Gymnasium's Humanoid Standup](https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#starting-state). +The starting state of the environment is the same as [Gymnasium's Humanoid Standup](https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#starting-state). @@ -123,6 +123,9 @@ All agent terminate and truncate at the same time, given the same conditions as ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). + - No longer observes `qfrc_actuator` of `root` & `cinert`, `cvel`, `qfrc_actuator`, `cfrc_ext` of `worldbody` (same as `Gymnasium/MuJoCo-v5/HumanoidStandup`). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Added/Fixed Global observations (The Humanoids's torso: `rootx`, `rooty`, `rootz`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_multiagentswimmer.md b/docs/envs/MaMuJoCo/ma_multiagentswimmer.md new file mode 100644 index 00000000..ce3657ca --- /dev/null +++ b/docs/envs/MaMuJoCo/ma_multiagentswimmer.md @@ -0,0 +1,81 @@ +--- +firstpage: +lastpage: +--- +ManySegmentSwimmer + +# Swimmer +```{figure} figures/many_segment_swimmer.png + :name: many_segment_swimmer +``` + + +This Environment is part of [MaMuJoCo](https://robotics.farama.org/envs/MaMuJoCo/) environments. Please read that page first for general information. +The task is variation of [Gymansium's MuJoCo/Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/), which instead of having 2 segments, it has configurable amount of segments. + +The task was first introduced by Christian A. Schroeder de Witt in ["FACMAC: Factored Multi-Agent Centralised Policy Gradients"](https://arxiv.org/abs/2003.06709) + + +## Action Space +The shape of the action space depends on the partitioning. The partitioning has the following form: `${Number Of Agents}x${Number Of Segments per Agent}` + +| Instantiate | `env = mamujoco_v1.parallel_env("ManySegmentSwimmer", ${Number Of Agents}x${Number Of Segments per Agent})` | +|-----------------------|------------------------------------------------------| +| Agents | `agents= ['agent_0', ..., 'agent_${Number Of Agents}']` | +| Number of Agents | `${Number Of Agents}` | +| Action Spaces | `{${agents} : Box(-1, 1, (${Number Of Segments per Agent},), float32)}` | +| Part partition | `(joint0, joint1,)` | + +The environment is partitioned in `${Number Of Agents}` parts, with each part corresponding to `${Number Of Segments per Agent}` joints. + +#### Agent 0 action space +| Num | Action | Control Min | Control Max | Name (in corresponding XML file) | Joint | Unit | +|-----|------------------------------------|-------------|-------------|----------------------------------|-------|--------------| +| 0 | Torque applied on the first rotor | -1 | 1 | motor1_rot | hinge | torque (N m) | +| 1 | Torque applied on the second rotor | -1 | 1 | motor2_rot | hinge | torque (N m) | +| ... | ... | -1 | 1 | ... | hinge | torque (N m) | +| `${Number Of Segments per Agent}` | Torque applied on the agent's last rotor | -1 | 1 | motor`${Number Of Segments per Agent}`_rot | hinge | torque (N m) | +#### Agent 1 action space +| Num | Action | Control Min | Control Max | Name (in corresponding XML file) | Joint | Unit | +|-----|------------------------------------|-------------|-------------|----------------------------------|-------|--------------| +| 0 | Torque applied on the `${Number Of Segments per Agent}` rotor | -1 | 1 | ??? | hinge | torque (N m) | +| ... | ... | -1 | 1 | ... | hinge | torque (N m) | +| `${Number Of Segments per Agent}` | Torque applied on the agent's last rotor | -1 | 1 | motor`2x${Number Of Segments per Agent}`_rot | hinge | torque (N m) | +#### Agent ... action space +... + + + +## Observation Space +| Observation Categories || +|-----------------------|------------------------------------------------------| +| Default `local_categories` | `[["qpos", "qvel"], ["qpos"]]` | +| Default `global_categories` | `("qpos", "qvel")` | +| Supported observation categories | `"qpos", "qvel"` | + +Besides the local observation of each agent (which depend on their parts of the agent, the observation categories and the observation depth), each agent also observes the position and velocity items of the swimmer's tip. +See more at the [Gymnasium's Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/#observation-space). + + + +## Rewards +All agents receive the same [Gymnasium's Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/#observation-space) reward. + + + +## Starting state +The starting state of the environment is the same as [Gymnasium's Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/#starting-state). + + + +## Episode End +All agent terminate and truncate at the same time, given the same conditions as [Gymnasium's Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/#episode-end). + + +## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). + - Now uses the same `option.timestep` as `Gymansum/Swimmer` (0.01). + - Updated model to work with `mujoco>=3.0.0`. +* v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). + diff --git a/docs/envs/MaMuJoCo/ma_pusher.md b/docs/envs/MaMuJoCo/ma_pusher.md index d4f92f30..1c45a1f2 100644 --- a/docs/envs/MaMuJoCo/ma_pusher.md +++ b/docs/envs/MaMuJoCo/ma_pusher.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Pusher](https://gymnasium.farama.org/environment ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/pusher.png :name: pusher ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Pusher", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Pusher", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -50,7 +50,7 @@ If partitioning, is None then the environment contains a single agent with the s :name: pusher_3p ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Pusher", "3p")`| +| Instantiate | `env = mamujoco_v1.parallel_env("Pusher", "3p")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 3 | @@ -98,7 +98,7 @@ All agents receive the same [Gymnasium's Pusher](https://gymnasium.farama.org/en ## Starting state -The starting state of the environment is the as [Gymnasium's Pusher](https://gymnasium.farama.org/environments/mujoco/pusher/#starting-state). +The starting state of the environment is the same as [Gymnasium's Pusher](https://gymnasium.farama.org/environments/mujoco/pusher/#starting-state). @@ -107,4 +107,6 @@ All agent terminate and truncate at same time given the same conditions as [Gymn ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), first implemented here. diff --git a/docs/envs/MaMuJoCo/ma_reacher.md b/docs/envs/MaMuJoCo/ma_reacher.md index 0bd613d9..a587e1c8 100644 --- a/docs/envs/MaMuJoCo/ma_reacher.md +++ b/docs/envs/MaMuJoCo/ma_reacher.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Reacher](https://gymnasium.farama.org/environmen ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/reacher.png :name: reacher ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Reacher", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Reacher", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -45,7 +45,7 @@ If partitioning, is `None`, then the environment contains a single agent with th :name: reacher_2x1 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Reacher", "2x1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("Reacher", "2x1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -84,7 +84,7 @@ All agents receive the same [Gymnasium's Reacher](https://gymnasium.farama.org/e ## Starting state -The starting state of the environment is the as [Gymnasium's Reacher](https://gymnasium.farama.org/environments/mujoco/reacher/#starting-state). +The starting state of the environment is the same as [Gymnasium's Reacher](https://gymnasium.farama.org/environments/mujoco/reacher/#starting-state). @@ -93,6 +93,8 @@ All agent terminate and truncate at the same time, given the same conditions as ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Added/Fixed Global observations (The Targets's coordinates: `targetx`, `targety`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_single.md b/docs/envs/MaMuJoCo/ma_single.md index dab60914..599ce7d8 100644 --- a/docs/envs/MaMuJoCo/ma_single.md +++ b/docs/envs/MaMuJoCo/ma_single.md @@ -20,7 +20,7 @@ The Purpose of these is to allow researchers to debug multi-agent learning algor ## Action Space -The action spaces is depended on the partitioning. +The shape of the action space depends on the partitioning. The partitioning has the following form: ## Observation Space The agent receives the same observations as the single agent Gymnasium environment. @@ -43,6 +43,8 @@ The agent terminates and truncates at the same time, given the same conditions a ## Version History -v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of [the original multiagent_mujuco](https://github.com/schroederdewitt/multiagent_mujoco). +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). +* v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of [the original multiagent_mujuco](https://github.com/schroederdewitt/multiagent_mujoco). No Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)). diff --git a/docs/envs/MaMuJoCo/ma_swimmer.md b/docs/envs/MaMuJoCo/ma_swimmer.md index 97400b5c..0d066615 100644 --- a/docs/envs/MaMuJoCo/ma_swimmer.md +++ b/docs/envs/MaMuJoCo/ma_swimmer.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Swimmer](https://gymnasium.farama.org/environmen ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/swimmer.png :name: swimmer ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Swimmer", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Swimmer", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -45,7 +45,7 @@ If partitioning, is `None` then the environment contains a single agent with the :name: swimmer_2x1 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Swimmer", "2x1")`| +| Instantiate | `env = mamujoco_v1.parallel_env("Swimmer", "2x1")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -83,7 +83,7 @@ All agents receive the same [Gymnasium's Swimmer](https://gymnasium.farama.org/e ## Starting state -The starting state of the environment is the as [Gymnasium's Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/#starting-state). +The starting state of the environment is the same as [Gymnasium's Swimmer](https://gymnasium.farama.org/environments/mujoco/swimmer/#starting-state). @@ -92,6 +92,8 @@ All agent terminate and truncate at the same time, given the same conditions as ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Added/Fixed Global observations (The Swimmer's front tip: `free_body_rot`) not being observed. diff --git a/docs/envs/MaMuJoCo/ma_walker2d.md b/docs/envs/MaMuJoCo/ma_walker2d.md index e273e5c3..cfa7ef03 100644 --- a/docs/envs/MaMuJoCo/ma_walker2d.md +++ b/docs/envs/MaMuJoCo/ma_walker2d.md @@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Walker2D](https://gymnasium.farama.org/environme ## Action Space -The action spaces is depended on the partitioning +The shape of the action space depends on the partitioning. The partitioning has the following form: ### if partitioning is None: ```{figure} figures/walker2d.png :name: walker2d ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Walker2D", None)` | +| Instantiate | `env = mamujoco_v1.parallel_env("Walker2D", None)` | |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0']` | | Number of Agents | 1 | @@ -49,7 +49,7 @@ If partitioning, is `None` then the environment contains a single agent with the :name: walker2d_2x3 ``` -| Instantiate | `env = mamujoco_v0.parallel_env("Walker2d", "2x3")`| +| Instantiate | `env = mamujoco_v1.parallel_env("Walker2d", "2x3")`| |-----------------------|------------------------------------------------------| | Agents | `agents= ['agent_0', 'agent_1']` | | Number of Agents | 2 | @@ -92,7 +92,7 @@ All agents receive the same [Gymnasium's Walker2D](https://gymnasium.farama.org/ ## Starting state -The starting state of the environment is the as [Gymnasium's Walker2D](https://gymnasium.farama.org/environments/mujoco/walker2d/#starting-state). +The starting state of the environment is the same as [Gymnasium's Walker2D](https://gymnasium.farama.org/environments/mujoco/walker2d/#starting-state). @@ -102,6 +102,9 @@ All agent terminate and truncate at the same time given the same conditions as [ ## Version History +* v1: + - Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572). + - Fixed bug: global nodes are now `[root_x, root_z, root_y]` (used to be `[root_x, root_x, root_z]`). - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco). Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)): - Added/Fixed Global observations (The Walker's top: `rootx`, `rooty`, `rootz`) not being observed. diff --git a/gymnasium_robotics/__init__.py b/gymnasium_robotics/__init__.py index 1b1c37d4..49c86374 100644 --- a/gymnasium_robotics/__init__.py +++ b/gymnasium_robotics/__init__.py @@ -3,9 +3,9 @@ from gymnasium_robotics.core import GoalEnv from gymnasium_robotics.envs.maze import maps -from gymnasium_robotics.envs.multiagent_mujoco import mamujoco_v0 +from gymnasium_robotics.envs.multiagent_mujoco import mamujoco_v1 -__version__ = "1.2.4" +__version__ = "1.3.0a1" def register_robotics_envs(): diff --git a/gymnasium_robotics/envs/multiagent_mujoco/__init__.py b/gymnasium_robotics/envs/multiagent_mujoco/__init__.py index cf745d64..07eb2b5d 100644 --- a/gymnasium_robotics/envs/multiagent_mujoco/__init__.py +++ b/gymnasium_robotics/envs/multiagent_mujoco/__init__.py @@ -3,12 +3,6 @@ from gymnasium_robotics.envs.multiagent_mujoco.coupled_half_cheetah import ( # noqa: F401 CoupledHalfCheetahEnv, ) -from gymnasium_robotics.envs.multiagent_mujoco.many_segment_ant import ( # noqa: F401 - ManySegmentAntEnv, -) -from gymnasium_robotics.envs.multiagent_mujoco.many_segment_swimmer import ( # noqa: F401 - ManySegmentSwimmerEnv, -) from gymnasium_robotics.envs.multiagent_mujoco.mujoco_multi import ( # noqa: F401 MultiAgentMujocoEnv, ) diff --git a/gymnasium_robotics/envs/multiagent_mujoco/assets/many_segment_ant.xml.template b/gymnasium_robotics/envs/multiagent_mujoco/assets/many_segment_ant.xml.template index 3b6b4eb8..8bfc3308 100644 --- a/gymnasium_robotics/envs/multiagent_mujoco/assets/many_segment_ant.xml.template +++ b/gymnasium_robotics/envs/multiagent_mujoco/assets/many_segment_ant.xml.template @@ -1,7 +1,13 @@ + -