Using Vision to Enable Robotic Random Bin Picking

The development of a robotic random bin picking system that translates to a real-world factory application has received attention for more than 30 years, and has been called by some the “Search for the Holy Grail.” Random bin picking refers to a system where vision-enabled robots locate, grasp, and move single parts from a bin of jumbled or randomly piled parts. In recent years, advances in processing speed, new algorithms, and significant engineering have combined to solve limited versions of the problem, but the more general problem remained.

Semi-Random vs. Really Random

Many of the limited solutions handle what the machine vision industry refers to as “semi-random” problems, where parts are not totally randomized, but loosely fitted in a bin. In some cases, parts are roughly stacked, are flat 2D parts, or they arrive in a single layer in the bin creating less variance in part orientation and height. Semi-random picking techniques typically include pattern matching, 2D or 2.5D part location, and a standard approach trajectory for grasping parts.

Truly randomized bins, where parts are situated underneath or on top of one another in any position, create an additional set of problems. Parts jumbled on top of one another increase the number of potential orientations of a part, reduce the visibility of part features used to recognize individual parts, and require additional potential paths for grasping parts. Realworld solutions need to overcome all of these obstacles, and in addition to solving the vision problem, ensure that the robot avoids colliding with other parts or the bin itself.

All vision guidance solutions tackle extremely difficult problems caused by the possibility of harsh lighting conditions. In addition, random bin picking must deal with increased shadowing and specularities caused by reflections from other parts, variation in appearance of the part based on its pose, varying degrees of occlusion by other parts in the pile, and the lack of (a large number of ) salient features for recognition — a typical characteristic of rough or unfinished parts that are inexpensive enough to toss together in a bin. Another particularly difficult problem is cascading layers of parts — parts that partially lay on top of one another in a weaving formation — where a vision system must be able to recognize a part as safe to grasp even while under other parts.

In addition to vision issues, there are significant challenges with planning grasping paths for robot grippers that avoid collisions with the bin and the rest of the pile within the robot’s range of movement. Robot tools must be developed to grip parts from various positions, and must grasp parts without colliding with parts above, below, or next to the part. The system must also complete the task as fast as or faster than current manual or semi-manual systems to be commercially viable to customers.