Adding vision to a collaborative robot can open a world of possibilities for automation applications. With a vision system, a robot can inspect parts, check specific features of a part, recognize a part to pick it up, count items, adjust its path using visual feedback, color sort, and so on. The breadth of applications requires careful consideration to ensure selection of the right technology for the job.

Two common applications for collaborative robots are pick-and-place systems and quality control. Upfront evaluation will help determine if these systems would benefit most from a vision system or an alternative.

Pick and Place

The feature in a pick-and-place application that most influences the decision between using a vision system or another option is the part presentation. The robot must know where the parts are to pick them up. Following are some options that will solve part localization for pick-and-place applications.

Note that this discussion is about the pick position, but the same factors apply to the place position. Depending on the system requirements when placing a part, a vision system may be needed to determine the proper place position. An example of this might be locating the box in which the part is to be placed.


Parts can be placed in trays or secured with fixtures, so they are always at a constant position relative to the robot. This is usually simple to implement and program because the robot only needs to learn those stationary positions. Some robot models come with palletizing programs that will help a user teach positions quickly. If this kind of setup already exists, then using vision is probably overkill.

Without such a setup, it's important to remember that, depending on the part, designing a fixture can be costly. Also, fixtures can get tricky to design if a lot of parts will be fed to the robot without recharging the jigs. In this case, as many parts as possible must be placed within the robot's reach. Another downside arises when several different parts must be picked by the robot. In this case, several different jigs may be needed, which would therefore add to the cost. In addition, changeover time needs to be considered when estimating the efficiency of the system.

Bowl Feeders

Bowl feeders are meant to take bulk items and singularize them such that one part at a time is presented to the robot, and the robot always picks this part from the same position. Once a setup like this is installed and adjusted correctly, it usually works great. There is just one pick position that needs to be programmed into the robot. However, this type of equipment can be quite expensive and hard to adjust depending on the shape of the parts being picked. Furthermore, most of the time a bowl feeder is limited to sorting only one kind of part. Switching between different parts will require adding another bowl feeding system to the robot cell.


Conveyors can be a good option for part presentation. Using a conveyor without a vision system requires a datum line or point of reference that the part will rest against such that the system will always pick that part from the same position or location. In this case, a presence sensor is probably still needed to let the robot know when the part has arrived at the picking location. Once it does, some robots have an option to pick up the part while the conveyor is moving, otherwise the conveyor must be stopped during the picking application and started it once it's done. If there isn't a datum line, or the parts are not always in the same position or at the same spot on the conveyor, a camera can locate the part on the conveyor.

Vision Systems

Two common applications for collaborative robots — pick-and-place and quality control — can both benefit from the addition of a vision system. This pick-and-place system includes vision capability.

A vision system is commonly used to find a part's location and orientation. There are many ways to use a vision system to accomplish part recognition. The parts can simply rest on a surface and a 2D camera will locate them. The parts can also constantly be moving on a conveyor and a fixed or robot-mounted camera can locate the parts. Or the robot can do a 3D scan of a surface and search for parts.

These options all have pros and cons. Vision is a good option if parts are frequently switched. A robot can typically be taught new parts to pick quite fast, and there are no additional hardware costs when using a vision system as opposed to jigs. Usually, the vision algorithm can simply be adjusted and the system is good to go. Different algorithms can be saved for different parts to easily be reused later.

Vision algorithms are stored on a PC versus taking up space in the plant to stock different jigs. Because the price for such a system will vary widely depending on the technology needed, it's important to get expert advice before buying. This helps avoid overkill on the specs that results in buying something expensive that won't fully be used.

Quality Control

For quality control applications, first identify which aspects of the part need to be examined to determine whether the part is good or bad. Then, looking at the different vision algorithms will reveal which visions systems match the system's needs and can perform the required quality inspection task. Of course, there are alternatives to using vision to perform quality checks. Following are some options and comparisons to the use of a vision system.

Coordinate Measuring Machine

The camera's interface can help users add vision to a collaborative robot.

One example is the need to validate the dimensions of a part coming out of a computer numerical control (CNC) machine. The most common system used is a coordinate measuring machine (CMM). In this case, the system touches different surfaces on the part to measure it. Designers usually want CMM machines with an accuracy of approximately 0.25 mm. To reach this level of accuracy with a vision system is currently difficult and probably overly expensive.

Assume that at least 3 pixels are needed to cover the accuracy value of 0.25 mm, and the system has a 150 mm × 150 mm field of view, which is large enough to cover the part. Approximately an 1800 × 1800-pixel camera is needed to do the job. And this does not consider any lighting challenges that may come with the images of shiny metal objects coming out of a CNC machine.

Eddy Current

This technique can be used to detect a part's position and its defects. Due to its high sensitivity, it will be able to detect small cracks, deformities, pits, etc. However, using eddy currents is limited to conductive materials, whereas using a vision system is not.

Vision System

Vision will be the best option when products on a production line are coming out on a conveyor and the system simply needs to check part presence and do some measurement analysis without extreme precision. The camera can take a snapshot of the part as it is moving on the conveyor, allowing for live quality checks on the part without having to remove it from the production process. This will eliminate the need to do batch sample testing. The following list of vision functions can help determine if vision is the best option for a quality inspection process.

Pattern Matching: System operators teach a few good parts and a few bad parts to the camera. The camera will set a pass/fail threshold. Then, when taking an image of a new part, it will indicate if this part matches the threshold that's been set. This is often used when it's necessary to check a part for deformities or for the presence of a specific object on the part (e.g., holes in a part or a cap on a bottle).

Color Matching: Some vision systems can confirm the color of an object.

Optical Character Recognition (OCR): This is often used to confirm some printing on the product, making sure it is visible or written correctly and consistently with what was supposed to be printed.

Measuring: There are various measuring methods available in vision systems that can be used to do measurement checks using images. For example, measuring the distance between edges or the distance between holes.

Filters: A system can detect visual defects (pits, holes, cracks, etc.) using various filtering techniques like blob analysis.

Counting: This is a bit like measuring, but counts the number of objects or holes that are present on a part.

While vision systems can add tremendous value to collaborative robots, they are not always the most appropriate solution. Basic knowledge of machine vision provides insight on choosing the correct device for any given application. It can also help smooth the integration of simple vision capabilities into collaborative robotic systems.

This article was written by Amanda Lee, e-marketing coordinator for Robotiq, Saint-Nicolas, Quebec, Canada. To download the company's eBook, “Vision Systems for Collaborative Robots,” visit here .