1 Google DeepMind     2 Georgia Institute of Technology   3 Meta AI

IROS & RA-L 2024
Taking Mobile Manipulators into the Real World Workshop (RSS), 2023

arXiv Blog Post

All videos shown are of autonomous navigation runs with Spot using a policy trained solely in simulated indoor environments.


Abstract

We present IndoorSim-to-OutdoorReal (I2O), an end-to-end learned visual navigation approach, trained solely in simulated short-range indoor environments, and demonstrates zero-shot sim-to-real transfer to the outdoors for long-range navigation on the Spot robot. Our method uses zero real-world experience (indoor or outdoor), and requires the simulator to model no predominantly-outdoor phenomenon (sloped grounds, sidewalks, etc). The key to I2O transfer is in providing the robot with additional context of the environment (i.e., a satellite map, a rough sketch of a map by a human, etc.) to guide the robot's navigation in the real-world. The provided context-maps do not need to be accurate or complete-- real-world obstacles (e.g., trees, bushes, pedestrians, etc.) are not drawn on the map, and openings are not aligned with where they are in the real-world. Crucially, these inaccurate context-maps provide a hint to the robot about a route to take to the goal. We find that our method that leverages Context-Maps is able to successfully navigate hundreds of meters in novel environments, avoiding novel obstacles on its path, to a distant goal without a single collision or human intervention. In comparison, policies without the additional context fail completely. Lastly, we test the robustness of the Context-Map policy by adding varying degrees of noise to the map in simulation. We find that the Context-Map policy is surprisingly robust to noise in the provided context-map. In the presence of significantly inaccurate maps (corrupted with 50% noise, or entirely blank maps), the policy gracefully regresses to the behavior of a policy with no context.


Short-Range Indoor Simulation Navigation


We train navigation policies for Spot entirely in simulated indoor environments using short-range trajectories (~8m).


Episode #1

Episode #2



Long-Range Outdoor Real-World Navigation


We zero-shot transfer the policy trained in simulation to the real-world outdoors.

Route #1


Satellite Map

Context Map

Overlay Map


Context-Map Policy

Total Distance Travelled: 65.19 m

No-Context Policy

Total Distance Travelled: 16.65 m


Route #2


Satellite Map

Context Map

Overlay Map


Context-Map Policy

Total Distance Travelled: 111.87 m

No-Context Policy

Total Distance Travelled: 11.46 m


Route #3


Satellite Map

Context Map

Overlay Map


Context-Map Policy

Total Distance Travelled: 130.09 m

No-Context Policy

Total Distance Travelled: 5.11 m

Citation

@inproceedings{truong2023i2o,
    title={IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors without any Outdoor Experience}, 
    author={Joanne Truong and April Zitkovich and Sonia Chernova and Dhruv Batra and Tingnan Zhang and Jie Tan and Wenhao Yu}, 
    booktitle={IEEE Robotics and Automation Letters},
    year={2024}
}
            

Related Projects

Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigation
Joanne Truong, Max Rudolph, Naoki Yokoyama, Sonia Chernova, Dhruv Batra, Akshara Rai
CoRL 2022, Spotlight at Sim-to-Real Robot Learning: Locomotion and Beyond Workshop
paper / project page