Problem about visualizing mujoco simulation when running:test_gym_env_human.py

[parzivar]

when i run: python franka_sim/franka_sim/test/test_gym_env_human.py

I am very interested in SERL， but i dont known how to solve this problem.
ubuntu20.04, Nvidia 565, cuda12.6,
I would be very grateful if you could reply me

[parzivar]

[claudio-dg]

Hi @parzivar, I get the exact same Error running it with Ubuntu 22.04 and Cuda 12.1.
Hope someone replies as i am looking forward to reproduce their Simulation.

[CeyaoZhang]

Also encountered the same problem with Ubuntu 22.04 + CUDA 12.6.
Looking for the reply orz !!!

[youliangtan]

Check if this issue is relevant to:

• Gymnasium AsyncVectorEnv for individual metaworld env Farama-Foundation/Metaworld#509
• Caught egl initial wrong when use multiprocessing google-deepmind/mujoco#991

Referred to here, on ubuntu setup, try adding multiprocessing.set_start_method('spawn') right after the import

[CeyaoZhang]

No, this error has nothing to do with operations like AsynVectorEnv or multiprocessing. The default code only concerns with one Env.

---

I also find that even run the following one line code

from franka_sim import envs
env = envs.PandaPickCubeGymEnv(action_scale=(0.1, 1))

will cause

Traceback (most recent call last):
File "/home/admin01/Projects/serl/franka_sim/franka_sim/test/test_gym_env_human.py", line 9, in
env = envs.PandaPickCubeGymEnv(action_scale=(0.1, 1))
File "/home/admin01/Projects/serl/franka_sim/franka_sim/envs/panda_pick_gym_env.py", line 148, in init
self._viewer.render(self.render_mode)
File "/home/admin01/anaconda3/envs/serl/lib/python3.10/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 706, in render
viewer = self._get_viewer(render_mode=render_mode)
File "/home/admin01/anaconda3/envs/serl/lib/python3.10/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 730, in _get_viewer
self.viewer = OffScreenViewer(
File "/home/admin01/anaconda3/envs/serl/lib/python3.10/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 155, in init
super().init(model, data, width, height, max_geom, visual_options)
File "/home/admin01/anaconda3/envs/serl/lib/python3.10/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 53, in init
self.viewport = mujoco.MjrRect(0, 0, width, height)
TypeError: init(): incompatible constructor arguments. The following argument types are supported:
1. mujoco._render.MjrRect(left: int, bottom: int, width: int, height: int)

Invoked with: 0, 0, None, None

After checking the code in panda_pick_gym_env.py, I doubt this result due to the missing inputs height and width in

serl/franka_sim/franka_sim/envs/panda_pick_gym_env.py

Lines 144 to 147 in 2f9f315

	self._viewer = MujocoRenderer(
	self.model,
	self.data,
	)

---

Looking forward your double check the sim code orz !!

[CeyaoZhang]

After setting

 self._viewer = MujocoRenderer(
            self.model,
            self.data,
            width=3,
            height=4
        )

the code works.

[CeyaoZhang]

Hi, @parzivar @claudio-dg , Can you check if this works for you too?
And I do suggest @parzivar change this issus's title.
I missed this issue several times because the original title was about "UserWarning", which was not a really problem.

[parzivar]

Thank you for your questions, and I sincerely thank @CeyaoZhang and @youliangtan for answering this question.
I have solved this problem by modifying the dependent version of the simulation environment, refer to 8ab8541
Finally, I made a fr3 robot teleoperation of an Xbox controller based on SERL. Interested friends can learn more about
https://github.com/parzivar/Franka-Research-3-Robot-Simulation-with-Xbox-Controller-Integration-in-MuJoCo

[CeyaoZhang]

@parzivar I check your method and it works.
I think this issue lies in the different gymnasium versions.

[parzivar]

@CeyaoZhang I have another question about SERL：#47
If you have encountered and solved it, I hope you can give me some guidance, thank you

[CeyaoZhang]

I don't figure out your problem. You may give me more details and we can discuss in #47.