Quastion about action distribution

[xiaotangzh]

Hi Chen, the MaskedMimic looks really robust, I am interested in reproducing your motion tracking model using Transformer encoder, may I ask why setting the diagonal covariance matrix of the action Gaussian to be fixed at exp(-2.9) rather than learnable parameters? Doesn't it produce jittery action because the deviation is large?

[tesslerc]

This is a good question.
Turns out that although a fixed gaussian distribution is very noisy during training (if you visualize the training you'll see how jittery it is), when you infer the trained policy and play the mean actions you get fairly smooth and natural behavior.

I think learnable variance can be useful, but adds another parameter to tune (entropy) and so far it feels that this isn't what's blocked us from scaling to more complex tasks. Happy to revisit this if you think it's useful!

[xiaotangzh]

Thank you Chen for sharing the experience! :)