AllSee: Battery-Free Gesture Recognition System for Electronic Devices

Gesture control for electronics could become an alternative to touchscreens and sensing technologies that consume a lot of power and only work when users can see their smartphones and tablets. University of Washington researchers have developed a gesture recognition system that runs without batteries and lets users control their electronic devices hidden from sight with simple hand movements. The low-cost prototype - called AllSee - uses existing TV signals as both a power source and the means for detecting a user's gesture command. The technology builds on previous work of leveraging Wi-Fi signals for gesture recognition around the home. For AllSee, the researchers built a sensor that can be placed on an electronic device like a smartphone. The sensor uses an ultra-low-power receiver to extract and classify gesture information from nearby wireless transmissions. When a person makes a hand gesture, it changes the amplitude of the wireless signals in the air. The AllSee sensors then recognize unique amplitude changes created by specific gestures.

Transcript

00:00:02 state-of-the-art gesture recognition systems consume significant power and computational resources which severely limit their applicable 'ti in this project we introduced all see the first gesture recognition system that can run on a range of devices even those without batteries all C can enable always-on gesture recognition and works even when the phone is in your pocket

00:00:26 also you leverages existing wireless signals such as TV transmissions for both power and means for detecting a user's gesture command as the human body moves the changes in the wireless signal is reflected by the body can be analyzed on the receiver let's consider a poll gesture as the user moves the arm away from the receiver the Wireless change is induced by the gesture decrease with

00:00:50 time all C uses the insight that motion at any location farther from the receiver results in a smaller signal change than from a closeby location all C encodes these unique changes with passive components such as resistors and capacitors to classify a rich set of gestures this enables operation on power levels as low as 30 micro watts here we have a user performing a series of

00:01:16 gestures the all C prototype detects and classifies these gestures in real-time and displays the results on the screen the signal variations corresponding to the various gestures can be seen on the scope all C consumes negligible power and Knable always-on gesture recognition to demonstrate this we integrate the all C prototype with an off-the-shelf mobile phone here we see the user interacting

00:01:40 with the mobile phone using the same rich set of gestures all C enables gesture recognition in a broad range of applications for instance all C can be used to control music on a mobile phone kept inside the pocket beyond mobile devices all seek and humanize and enable interaction with Internet of Things devices this

00:02:22 technology provides a seamless way to interact with ubiquitous devices such as home monitoring solutions all si can also open up opportunities for power efficient human robot interaction that can work even when the robot is not inside of the human beyond the examples shown in this video we believe all si can extend gesture recognition to a variety of devices independent of their

00:02:44 power constraints