[Question] agent_obsk in mujoco_multi.py

[xihuai18]

Question

Gymnasium-Robotics/gymnasium_robotics/envs/multiagent_mujoco/mujoco_multi.py

Lines 93 to 98 in b1acee9

	agent_obsk: Number of nearest joints to observe,
	If set to 0 it only observes local state,
	If set to 1 it observes local state + 1 joint over,
	If set to 2 it observes local state + 2 joints over,
	If it set to None the task becomes single agent (the agent observes the entire environment, and performs all the actions)
	The Default value is: 1

agent_obsk in the original repo controls only the observation construction, but not the action factorization. I am wondering why implementing agent 0 performs all the actions when agent_obsk is None here.

[Kallinteris-Andreas]

The behavior is the same as the original repo when agent_obsk=None, it simply becomes a single agent environment, it was just missdocumented

https://github.com/schroederdewitt/multiagent_mujoco/blob/b212ddd74b258e7cea006ff1d642b5ffada4b99d/multiagent_mujoco/mujoco_multi.py#L43

https://github.com/schroederdewitt/multiagent_mujoco/blob/b212ddd74b258e7cea006ff1d642b5ffada4b99d/multiagent_mujoco/mujoco_multi.py#L67

[xihuai18]

I think the main difference is

https://github.com/schroederdewitt/multiagent_mujoco/blob/b212ddd74b258e7cea006ff1d642b5ffada4b99d/multiagent_mujoco/mujoco_multi.py#L108-L116

and

Gymnasium-Robotics/gymnasium_robotics/envs/multiagent_mujoco/mujoco_multi.py

Lines 296 to 297 in b1acee9

	if self.agent_obsk is None:
	return actions[self.possible_agents[0]]

In the original repo, the final action to use is constructed by multiple agents, while the final action is decided by the first agent in this repo, when agent_obsk == None.

[xihuai18]

I make a PR #228 and pass my tests in https://github.com/xihuai18/MaMuJoCo-PettingZoo/blob/main/tests/mamujoco_pettingzoo_v1_test.py, I think these changes make the current implementation perform the same behavior as the original repo.

[Kallinteris-Andreas]

1. Can you provide an example of the behavior of agent_obsk=None when using the original implementation

>>> import gymnasium_robotics         
>>> from gymnasium_robotics import mamujoco_v1
>>> env = mamujoco_v1.parallel_env("Ant", "2x4", agent_obsk=None)
>>> env.action_spaces
{'agent_0': Box(-1.0, 1.0, (8,), float32)}
>>> env.observation_spaces
{'agent_0': Box(-inf, inf, (105,), float64)}

2. regardless of (1.), what is the reason you want fully observable multi-agent factorizations of the environments, this is equivalent to a single agent environment, with extra steps (https://discord.com/channels/765294874832273419/808462033866588180/1261698405630214185 )