IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors
without any Outdoor Experience
1 Google DeepMind 2 Georgia Institute of Technology 3 Meta AI
IROS & RA-L 2024
Taking Mobile Manipulators into the Real World Workshop (RSS), 2023
All videos shown are of autonomous navigation runs with Spot using a policy trained solely in simulated indoor environments.
Abstract
We present IndoorSim-to-OutdoorReal (I2O), an end-to-end learned visual navigation approach, trained solely in simulated short-range indoor environments, and demonstrates zero-shot sim-to-real transfer to the outdoors for long-range navigation on the Spot robot. Our method uses zero real-world experience (indoor or outdoor), and requires the simulator to model no predominantly-outdoor phenomenon (sloped grounds, sidewalks, etc). The key to I2O transfer is in providing the robot with additional context of the environment (i.e., a satellite map, a rough sketch of a map by a human, etc.) to guide the robot's navigation in the real-world. The provided context-maps do not need to be accurate or complete-- real-world obstacles (e.g., trees, bushes, pedestrians, etc.) are not drawn on the map, and openings are not aligned with where they are in the real-world. Crucially, these inaccurate context-maps provide a hint to the robot about a route to take to the goal. We find that our method that leverages Context-Maps is able to successfully navigate hundreds of meters in novel environments, avoiding novel obstacles on its path, to a distant goal without a single collision or human intervention. In comparison, policies without the additional context fail completely. Lastly, we test the robustness of the Context-Map policy by adding varying degrees of noise to the map in simulation. We find that the Context-Map policy is surprisingly robust to noise in the provided context-map. In the presence of significantly inaccurate maps (corrupted with 50% noise, or entirely blank maps), the policy gracefully regresses to the behavior of a policy with no context.
Short-Range Indoor Simulation Navigation
We train navigation policies for Spot entirely in simulated indoor environments using short-range trajectories (~8m).
Episode #1
Episode #2
Long-Range Outdoor Real-World Navigation
We zero-shot transfer the policy trained in simulation to the real-world outdoors.
Route #1
Satellite Map
Context Map
Overlay Map
Context-Map Policy
Total Distance Travelled: 65.19 m
No-Context Policy
Total Distance Travelled: 16.65 m
Route #2
Satellite Map
Context Map
Overlay Map
Context-Map Policy
Total Distance Travelled: 111.87 m
No-Context Policy
Total Distance Travelled: 11.46 m
Route #3
Satellite Map
Context Map
Overlay Map
Context-Map Policy
Total Distance Travelled: 130.09 m
No-Context Policy
Total Distance Travelled: 5.11 m
Citation
@inproceedings{truong2023i2o,
title={IndoorSim-to-OutdoorReal: Learning to Navigate Outdoors without any Outdoor Experience},
author={Joanne Truong and April Zitkovich and Sonia Chernova and Dhruv Batra and Tingnan Zhang and Jie Tan and Wenhao Yu},
booktitle={IEEE Robotics and Automation Letters},
year={2024}
}
Related Projects
Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigation
Joanne Truong,
Max Rudolph,
Naoki Yokoyama,
Sonia Chernova,
Dhruv Batra,
Akshara Rai
CoRL 2022, Spotlight at Sim-to-Real Robot Learning: Locomotion and Beyond Workshop
paper / project page