Not normalizing the expert states when computing gradient penalty.

[EGalahad]

AMP_for_hardware/rsl_rl/rsl_rl/algorithms/amp_ppo.py

Line 235 in bfb0dbd

grad_pen_loss = self.discriminator.compute_grad_pen(

When using state normalization, the sample_amp_expert tuple is not normalized.

[EGalahad]

AMP_for_hardware/rsl_rl/rsl_rl/algorithms/amp_ppo.py

Line 254 in bfb0dbd

if self.amp_normalizer is not None:

and I think maybe we should update the normalizer stats before normalizing them?

[apirrone]

@EGalahad What effect do you think this has ?

[EGalahad]

I just think it is weird to pass in the normalized values to update the normalizer. I think you should pass un-normalized version so that the normalizer can update running mean?