[Question] Error in multienv GPU setup with "pd_ee_target_delta_pose"

[rechim25]

Hi, I'm having an error when running multi-env setup on the GPU. When I set control_mode to "pd_ee_target_delta_pose" I get the following error:

File "/home/rechim/anaconda3/envs/mani/lib/python3.10/site-packages/mani_skill/agents/controllers/pd_ee_pose.py", line 44, in _initialize_joints
    self._check_gpu_sim_works()
  File "/home/rechim/anaconda3/envs/mani/lib/python3.10/site-packages/mani_skill/agents/controllers/pd_ee_pose.py", line 186, in _check_gpu_sim_works
    self.config.use_target == False
AssertionError: Currently cannot take actions relative to last target pose in GPU sim

This is a code to reproduce the exception:

import gymnasium as gym
import numpy as np
import time
from mani_skill.envs.sapien_env import BaseEnv

NUM_ENVS = 5

if __name__ == '__main__':    
    env: BaseEnv = gym.make(
        id='PickCube-v1',
        num_envs=NUM_ENVS,
        obs_mode='pointcloud',
        control_mode='pd_ee_target_delta_pose',
        reward_mode='sparse',
        enable_shadow=True,
        sim_backend='gpu',
    )

    env.reset(seed=0)

    for i in range(500):
        random_action = np.random.rand(NUM_ENVS, 8)
        obs, _, _, _, info = env.step(action=random_action)
        time.sleep(0.1)

I'm wondering if there is any way to setup use_target = False on the agent so I can run in "pd_ee_target_delta_pose" control mode?

Also, the documentations mentions that At the moment we only support fast GPU accelerated delta end-effector control.

[StoneT2000]

If possible I recommend using pd_ee_delta_pose instead of pd_ee_target_delta_pose. I think you might mean use_target=True? as use_target=False is the setting of pd_ee_delta_pose for the panda robot (the default for PickCube-v1)

pd_ee_target_delta_pose applies delta pose actions to a target which is accumulated over time. However this method of action control on the GPU is not implemented at the moment, it is a little bit complicated since I would need to write one of the CPU IK algorithms on the GPU. I plan to do so soon as this is part of efforts for GPU accelerated real2sim simpler-env/SimplerEnv#35 (which uses pd ee target delta pose controllers).

[StoneT2000]

ok this method control is now supported on GPU. You can use any of the IK based controllers just like on CPU sim.