Using Vision to Enable Robotic Random Bin Picking

Sunday, June 01 2008

Methods and Approaches

Research focused on visually recognizing and locating 3D objects uses either 2D data from a single image or 3D data from stereo images or range scanners. Methods can be subdivided into model-based, appearance-based, and 3D data approaches.

The model-based approaches suffer from difficulties in feature extraction under harsh lighting conditions. Typical parts will not contain a large number of features, limiting the accuracy of a model-based fit to noisy image data. Appearancebased approaches have problems in segmenting out the object for recognition, have trouble with occlusion, and may not provide a 3D pose accurate enough for grasping purposes.

Approaches with 3D data face lighting effects that cause problems for stereo reconstruction, and specularities that create spurious data both for stereo and laser range finders. Once the 3D data is generated, there are the issues of image segmentation and representation. On the representation side, more complex models are often used than in the 2D case. These models contain a larger number of free parameters, which can be difficult to fit to noisy data.

A Vision Solution

A solution that is showing great promise combines the strongest aspects of both 2D and 3D object recognition. First, a robot-mounted stereo camera is used to acquire images from multiple planned viewpoints above the bin in order to increase the probability of observing parts in desirable positions. This method of continually moving the camera to new viewpoints is called Active Vision — an intelligent sensing strategy that acquires and uses multiple image information for a given task. Active Vision takes full advantage of a robot-mounted camera by moving from areas with too much reflection, shadowing, or occlusions, to more suitable areas where better source images can be gathered to improve both 2D and 3D algorithms.

Next, images from one of the robot-mounted stereo cameras are analyzed using 2D matching algorithms to locate potential part candidates. The 3D matching stage then compares a 3D model of the part, called a 3D point cloud, against the runtime 3D surface map of the pile in the area chosen by the 2D matching stage. The 3D surface map data is determined using both the stereo cameras. Once parts are located, the current best candidate for picking is chosen based on various criteria. Three-dimensional models of the robot arm, end-of-arm tool, and the bin are used to determine possible collisions with known part locations, and the 3D surface map in the region around the grasp points is tested to ensure there is nothing obstructing the gripper as it moves to grasp the part.

Random Bin Picking™ (RBP™ ) technology, developed by Braintech, only considers cases where there is a single part type to be recognized; however, the part may exhibit specularity and may contain only a small number of significant features that are useful for recognition. A focus on edge detection and use of Active Vision enables the system to be more resistant to changes in lighting. The system deals with occlusion by simply not requiring a complete match. The number of found part features required at a minimum is part-specific, but generally stays in the 60 to 70% range of total features.

Random bin picking streamlines processes along assembly lines by removing the need for expensive custom crates, fixturing, precision feeding mechanisms in the case of automated systems, and ergonomically dangerous manual labor in the case of human-operated part feeding. As a result of reduced fixturing costs and improved ergonomics, RBP represents a significant return on investment.

This article was written by Kait Jones, Marketing Coordinator, and Jeff Beis, Robot Vision Scientist, at Braintech, Inc., Vancouver, BC, Canada.
For more information visit http://info.hotims.com/ 15132-155.