Gaze-Tracking Glasses Control Drone to Fly Wherever You Look

Researchers from the University of Pennsylvania, U.S. Army Research Laboratory, and New York University have developed a pair of lightweight gaze-tracking glasses and a small computing unit, along with a small drone that will fly wherever you look. The glasses are equipped with a gaze tracker, a camera, and an Inertial Measurement Unit (IMU). These components estimate the relative position of the head with respect to a quadroter and decouple the gaze direction from the head orientation.

Transcript

00:00:04 in this work we addressed the problem of providing human assisted quadrotor navigation using a set of eye tracking glasses the advent of these kinds of devices provides the opportunity to create new non-invasive forms of interactions between humans and robots [Music] [Music] we show how a set of glasses equipped

00:00:42 with a gaze tracker camera and inertial measurement unit or IMU can be used to estimate the relative position of the human with respect to the quadrotor and allow the human to send new 3d navigation waypoints to the robot in an instrument free environment we combined camera and IMU data to track the human's head orientation which allows us to decouple the gaze direction

00:01:12 from the head motion we train and use a deep neural network to detect the flying robot we evaluate the proposed approach experimentally and show that our pipeline is able to successfully achieve human guided autonomy for spatial tasking [Music] the proposed approach can be employed in a wide range of scenarios including

00:01:40 inspection and first response and it can be used by people with disabilities that affect their mobility [Music] [Music]