'RF-Grasp' Robot Finds and Grasps Hidden Objects

“We’re trying to give robots superhuman perception,” said MIT Associate Professor Fadel Adib.  How exactly? Adib and a team of MIT researchers are combining vision with radio frequency (RF) sensing so that robots can find and grasp objects, even when they are hidden. See how the "RF-Grasp" technology can be used in warehouses and manufacturing operations.

Transcript

00:00:02 This video presents RF-Grasp, a system from MIT that  enables robotic grasping of hidden  objects. For example, here the robot needs to  find and grasp a target object that is under  clutter. As you can see, the robot’s camera’s  line of sight is blocked and the object is not in the  camera’s view. I will show you how our system  can explore the environment, to approach the  object, de-clutter its vicinity, and pick it up Our system relies on RF perception, using the antennas you see here.   The target object is tagged with a very cheap RFID   sticker. RFID tags such as this  one are on billions of objects. 

00:00:52 Unlike visible light, RF signals can traverse through occlusion.   This allows our system to locate and identify the object,   as you can see in the bottom left. In contrast, the system cannot see the  object behind an occlusion using the camera.  By fusing camera visual data with RF perception,  the robot can maneuver efficiently around  obstacles and move toward the object. Once it  determines that the object is within its reach, it  moves to a second phase, which is called RF  visual grasping. Although it cannot see the target  object, it knows where it is due to the RF  perception. This allows the system to de-clutter the  object’s vicinity. It then grasps the target object, and declares task completion,  but it knows the 

00:01:42 target object has moved, as  you can see in the bottom left.  RF Grasp can perform more complex tasks, like  selective sorting. Here, it needs to identify and  locate all objects that belong  to a certain semantic class,   and put them into a sorting bin. Using RF Perception, it knows where all of the objects   are located, and follows similar steps to extract each object.   It localizes an object from the requested  category, moves above it, de-clutters the vicinity, and then  grasps it and puts it in the bin.  As you can see in the bottom left, the robot can  track the object’s location. This is important,  because it allows the robot to close the loop  and determine if it has missed an object that it 

00:02:33 was attempting to grasp and needs  to try again, or if it has picked  up the right item. When all of the  objects from the requested category are extracted  and are in the sorting bin, it declares task  completion. To learn more about our system   please read our paper or visit our website at rfgrasp.mit.media.edu. Thanks for listening!