Researchers have developed a robot that uses radio waves, which can pass through walls, to sense occluded objects. The robot, called RF-Grasp, combines this powerful sensing with more traditional computer vision to locate and grasp items that might otherwise be blocked from view. The advance could one day streamline warehouse operations or help a machine pluck a screwdriver from a jumbled toolkit.
Warehouse work is still usually the domain of humans, not robots, despite sometimes dangerous working conditions. That’s in part because robots struggle to locate and grasp objects in such a crowded environment. Using optical vision alone, robots can’t perceive the presence of an item packed away in a box or hidden behind another object on the shelf because visible light waves don’t pass through walls — but radio waves can.
Radio frequency (RF) identification systems have two main components: a reader and a tag. The tag is a tiny computer chip that gets attached to or, in the case of pets, implanted in the item to be tracked. The reader then emits an RF signal, which gets modulated by the tag and reflected back to the reader. The reflected signal provides information about the location and identity of the tagged item.
RF-Grasp uses both a camera and an RF reader to find and grab tagged objects, even when they’re fully blocked from the camera’s view. It consists of a robotic arm attached to a grasping hand. The camera sits on the robot’s wrist.
The RF reader stands independent of the robot and relays tracking information to the robot’s control algorithm. So, the robot is constantly collecting both RF tracking data and a visual picture of its surroundings.
Integrating these two data streams into the robot’s decision making was one of the biggest challenges the researchers faced. The robot has to decide, at each point in time, which of the streams is more important to think about. The robot initiates the seek-and-pluck process by pinging the target object’s RF tag for a sense of its whereabouts. The sequence is akin to hearing a siren from behind, then turning to look and get a clearer picture of the siren’s source.
With its two complementary senses, RF-Grasp zeroes in on the target object. As it gets closer and even starts manipulating the item, vision, which provides much finer detail than RF, dominates the robot’s decision making.
Compared to a similar robot that was equipped with only a camera, RF-Grasp was able to pinpoint and grab its target object with about half as much total movement. Plus, RF-Grasp displayed the unique ability to “declutter” its environment — removing packing materials and other obstacles in its way in order to access the target.
RF-Grasp’s RF sensing could instantly verify an item’s identity in a warehouse without the need to manipulate the item, expose its barcode, then scan it. Potential home applications including locating the right wrench from a toolbox or locating lost items.