Benchmarking Domain Randomisation for Visual Sim-to-Real Transfer

Abstract: Domain randomisation is a very popular method for visual sim-to-real transfer in robotics, due to its simplicity and ability to achieve transfer without any real-world images at all. But a number of design choices must be made to achieve optimal transfer. In this paper, we perform a large-scale benchmarking study on these choices, with two key experiments evaluated on a real-world object pose estimation task, which is also a proxy for end-to-end visual control. First, we study the quality of the rendering pipeline, and find that a small number of high-quality images is superior to a large number of low-quality images. Second, we study the type of randomisation, and find that both distractors and textures are important for generalisation to novel environments.

[PDF]