Sim2Real¶

What’s Sim2Real¶

The interest and appeal of simulation to reality transfer stems from the risks and costs associated with poor performance during training. Crashing in real will break physical hardware, while inefficient control can overheat motors and strain other electrical components. Due to the current era of compute, the safe, low-cost, and fast training that simulation provides has made the topic popular in research.

Some examples of successful sim2real transfer include the 1 system of Sadeghi and Levine, deployed to a collision avoiding drone; the paper by Muller et al. 2, with their remote-controlled cars that leveraged modularity and abstraction in the control policy; the results of Tan et al. 3, who trained a quadruped to have robust locomotion in real by leveraging domain randomization; and finally, Mysore et al. 4, with a framework that regularizes the objective function of the algorithm to promote smooth control around neighbouring and sequential observation states.

Why Sim2Real¶

Sim2Real draws its appeal from the fact that it is cheaper, safer and more informative to perform experiments in simulation than in the real world.

1: Fereshteh Sadeghi and Sergey Levine. Cad2rl: real single-image flight without a single real image. arXiv preprint arXiv:1611.04201, 2016.
2: Matthias Müller, Alexey Dosovitskiy, Bernard Ghanem, and Vladlen Koltun. Driving policy transfer via modularity and abstraction. arXiv preprint arXiv:1804.09364, 2018.
3: Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332, 2018.
4: Siddharth Mysore, Bassel Mabsout, Renato Mancuso, and Kate Saenko. Regularizing action policies for smooth control with reinforcement learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA), 1810–1816. IEEE, 2021.