'VISTA' Simulation Engine: End-to-End Autonomous Vehicle Control

End-to-end trained neural networks for autonomous vehicles have shown great promise for lane stable driving, but they lack methods to learn robust models at scale and require vast amounts of training data. In a paper published in IEEE Robotics and Automation Letters , MIT researchers present a data-driven simulation and training engine that can learn end-to-end autonomous vehicle control policies using only sparse rewards. By leveraging human-collected trajectories through a real environment, the team can render novel training data that allows virtual agents to drive along new local trajectories consistent with the road appearance and semantics. The team demonstrates the ability of control policies learned within the VISTA simulator to generalize to and navigate in previously unseen real-world roads, without access to any human control labels during training.