[Question] Sim2Sim Transfer Problem — Good Performance in Lab, Failure in Gazebo

[Renyu-ha]

Question

Hi all,

I'm encountering a sim2sim transfer issue. I've trained multiple policies in a custom lab simulation environment where all of them perform well. However, when transferring to Gazebo, the robot fails to even stand up.

I’ve already checked:
The scaling factors
The controller execution order
The observation list
The torque computation logic
And so on

Everything seems correct, but the issue still persists.
Does anyone know what else I could try or check to resolve this problem?

Thanks in advance!

[Renyu-ha]

The network architecture is simple.
The observations are common in locomotion tasks
No historical observations are used.

[x714543179]

我也遇到这个问题，可能是什么原因？

[hulehulema]

The network architecture is simple. The observations are common in locomotion tasks No historical observations are used.

This Project observations:["commands,"ang_vel","gravity_vec","dof_pos","dof_vel","actions"], I use this observations to train in isaacgym.When i using the train data sim2sim, there is a error,mat1 and mat2 shapes connot be multiplied(1x270 and 45x512).I guess isthe observations_history:[5,4,3,2,1,0], when i delete observations_history, there is no error, but the robot cannot work.So, what does observations_history meaning? How can i solve this problem?

[XuaaanQWQ]

Same. I trained in Himloco and checked everything. The standup state works, but failed when switch to rl state. The policy is robust enough to be deployed in the real robot, but failed in gazebo, which is really confusing.

[hulehulema]

Try changing the ovservations order.observations:["commands","ang_vel","gravity_vec","dof_pos","dof_vel","actions"] to ["ang_vel","gravity_vec","commands","dof_pos","dof_vel","actions"].

[XuaaanQWQ]

My training obs order is ["commands","ang_vel","gravity_vec","dof_pos","dof_vel","actions"], as well as rechecked the observations_history order. In fact, I use the same robot.py code in Isaac Gym to train a policy on Go2 and successfully deployed on Go2 in rl_sar, but when it comes to Lite3, it failed.
Everything seems correct, but the issue still persists.
There might be something about the description file, cuz I gennerally handwritten it. But i use the same urdf file to train the policy, and the structure works, I don't know where to check...........

[hulehulema]

My training obs order is ["commands","ang_vel","gravity_vec","dof_pos","dof_vel","actions"], as well as rechecked the observations_history order. In fact, I use the same robot.py code in Isaac Gym to train a policy on Go2 and successfully deployed on Go2 in rl_sar, but when it comes to Lite3, it failed. Everything seems correct, but the issue still persists. There might be something about the description file, cuz I gennerally handwritten it. But i use the same urdf file to train the policy, and the structure works, I don't know where to check...........

can you show me the env.py code.
I use UNITREE_RL_GYM to train the go2 with urdf.And rewrite the go2_env.py.The following code.
In the config.yaml,I delete the observations_history:[5,4,3,2,1,0]. It works well in gazebo.

from legged_gym.envs.base.legged_robot import LeggedRobot

from isaacgym.torch_utils import *
from isaacgym import gymtorch, gymapi, gymutil
import torch

class Go2Robot(LeggedRobot):
def _get_noise_scale_vec(self, cfg):
noise_vec = torch.zeros_like(self.obs_buf[0])
self.add_noise = self.cfg.noise.add_noise
noise_scales = self.cfg.noise.noise_scales
noise_level = self.cfg.noise.noise_level

    noise_vec[:3] = noise_scales.ang_vel * noise_level * self.obs_scales.ang_vel
    noise_vec[3:6] = noise_scales.gravity * noise_level
    noise_vec[6:9] = 0. # commands
    noise_vec[9:9+self.num_actions] = noise_scales.dof_pos * noise_level * self.obs_scales.dof_pos
    noise_vec[9+self.num_actions:9+2*self.num_actions] = noise_scales.dof_vel * noise_level * self.obs_scales.dof_vel
    noise_vec[9+2*self.num_actions:9+3*self.num_actions] = 0. # previous actions
    #if self.cfg.terrain.measure_heights:
    #    noise_vec[9+3*self.num_actions:270] = noise_scales.height_measurements *noise_level*self.obs_scales.height_measurements

    return noise_vec

def compute_observations(self):
    self.obs_buf = torch.cat((  self.base_ang_vel  * self.obs_scales.ang_vel,
                                self.projected_gravity,
                                self.commands[:, :3] * self.commands_scale,
                                (self.dof_pos - self.default_dof_pos) * self.obs_scales.dof_pos,
                                self.dof_vel * self.obs_scales.dof_vel,
                                self.actions,),dim=-1)

    self.privileged_obs_buf = torch.cat((  self.base_lin_vel * self.obs_scales.lin_vel,
                                self.base_ang_vel  * self.obs_scales.ang_vel,
                                self.projected_gravity,
                                self.commands[:, :3] * self.commands_scale,
                                (self.dof_pos - self.default_dof_pos) * self.obs_scales.dof_pos,
                                self.dof_vel * self.obs_scales.dof_vel,
                                self.actions
                                ),dim=-1)

    if self.add_noise:
        self.obs_buf += (2 * torch.rand_like(self.obs_buf) - 1) * self.noise_scale_vec

[XuaaanQWQ]

My problem ends up to the mismatch in Joint mapping sequence. This bug have been fixed in the newly published V3.1.0, so you can update the code and check whether problems still exist.

[hulehulema]

My problem ends up to the mismatch in Joint mapping sequence. This bug have been fixed in the newly published V3.1.0, so you can update the code and check whether problems still exist.

I don't know how to train go2 with observations_history in UNITREE_RL_GYM .There are no observations_historys. So I need the env.py to learn how to improve my model.
Can you show where is the observations_history in the code?