The ability to automatically extract information from digital images opens many doors for manufacturers striving to cut costs, improve quality, and streamline their processes overall. The primary uses of machine vision are inspection, gauging, and robot guidance. These tasks require repetitive activities to be performed as quickly as possible and with a high degree of accuracy, which is precisely where machine vision systems excel. They have also become increasingly cost-effective and simpler to implement over the last couple of decades.

Figure 1. The HAWK MV-4000 smart camera inspects a line of printed circuit boards.

Automated inspection is usually performed in-line, i.e. in tandem with items moving along the production line. Although the technology is complex, the basic sequence of using machine vision to perform automation tasks is simple: the vision system is triggered to capture an image when the part is in front of it, and then the system processes the image and extracts key features and data. Finally, it communicates the resulting data, which can be as simple as Pass/Fail, so that actions and decisions can be made. Such actions include triggering a reject mechanism or giving the locations of all parts for a robot to pick up.

Production lines move quickly — and the faster they can move, the better, as long as quality is maintained. For this reason, machine vision systems need to be able to almost instantaneously extract relevant information about the parts flying past them. In general, the highest processing speeds are reached by hooking up multiple cameras to a PC, but recent advances in smart camera technology have made single, self-contained devices almost as powerful as multi-camera systems (Figure 1).

In addition to awe-inspiring processing speed, proper illumination is essential on the production line to ensure that the camera “sees” the key features it is trying to process and ignores irrelevant details (such as bumps on a surface carrying a date stamp). Illumination requirements depend heavily on the nature of the inspection task. Trying to inspect a date stamp on a rough surface requires lighting that minimizes the appearance of surface irregularities, whereas trying to find dents on what should be a smooth surface requires lighting that specifically highlights these irregularities.

With advanced optics, superior illumination technology, and state-of-the-art algorithms, machine vision systems can perform a wide array of industrial automation tasks, including product inspection, print inspection, surface defect detection, gauging, and robot guidance.

A Foundational Toolkit

Even with near-perfect lighting on the production line, there is still bound to be plenty of variation. How can machine vision deal with so much irregularity and still perform its tasks in a reliable manner? As it turns out, the extensive capabilities of machine vision boil down to just a few fundamental functions: location, measurement, counting, and decoding. These four tasks depend on the common capabilities of recognizing pixel intensity values, finding edge contours, and matching shapes and patterns. These are complex tasks that involve comparing an image to a template pixel-by-pixel or comparing a set of perceived edges (generated by calculating the gradient of neighboring pixel intensities) to the edges of a template. There are many ways that pattern matching is performed and optimized, but the focus here is the resulting functionality that industrial professionals can use.

Basic Function: Location

In many cases, complex machine vision processing tasks begin with object location. The system is trained to recognize a specific pattern that it will then locate in various images featuring a variety of backgrounds. A common example of something a machine vision system would be expected to locate repeatedly is a two-dimensional barcode such as a data matrix. The system must find a barcode before it can decode it, and it must often do this extremely quickly as products bearing the codes fly by at high speeds. In fact, high-performance smart cameras can inspect up to 4,000 parts per minute, or the equivalent of a conveyor belt moving at 300 inches per second. Specific aspects of the codes indicate to the vision system that a barcode is present, as do other graphics that typically reside near barcodes, such as logos.

Matching specified patterns within the field of view also makes it possible to detect defects based upon how closely the located object corresponds to the pattern supplied for training. If the image of an item has a significant amount of deviation from the expected pattern — for instance, the presence of pixels not matching the expected color, or edges in places that shouldn't have edges — it will be flagged for rework or rejected from the production line.

Basic Function: Counting

Figure 2: A part is inspected for the correct number and placement of holes.

Once a machine vision system can locate an object of interest, it can count the number of similar objects present in an image. It can also tell whether the object is not present in an image, so a presence/absence test also falls under the category of counting. In the automotive industry, for example, the counting function is used to determine whether a component has the proper number of machined holes (Figure 2). Components with the wrong number or size of holes will be removed from the production line instead of becoming part of a new car.

Basic Function: Measurement

Machine vision can also calculate the distances between objects that it has located — a task known as gauging. It makes exceptionally precise measurements to verify that the distance between two components of a product — such as the bottom of a medication vial and the point to which the bottle is filled — meet expectations. Before the vision system can perform measurements, it must first be calibrated. Simple calibration involves telling the system that a unit of measure equals a certain number of pixels. More complex calibration involves showing the system a grid pattern wherein the grid spacing equals a known value. This ensures that measurements will be good even if the camera is tilted or rotated, or if there is distortion in the lens.

No physical manipulation of the objects on a production line is necessary to take measurements using vision, so there's no associated risk of damaging parts as there is with contact gauging. For this reason, machine vision gauging has virtually replaced contact gauging now that machine vision systems cost less to set up and require less programming knowledge. They can perform thousands of measurements per second, including tricky calculations like circularity that contact gauging has a hard time with. (This is because contact gauging is limited to a few points of contact, whereas vision can quickly and easily utilize up to 360 points on a circle to get a much better answer.)

Basic Function: 1D and 2D Decoding

In many industries, especially consumer electronics and automotive, all parts are marked with machine-readable codes. It is an essential function of the machine vision system that it can decode these symbologies so that the parts can be tracked throughout the production process, and documented as to which finished product they become part of. This is a concept known as Track and Trace. This information can be recorded as historical data in case questions arise at a later date, such as when trying to determine what limited set of products may need to be recalled because of a faulty component.

By combining these basic functions — location, counting, measurement, and decoding — in various ways, industrial automation systems can take care of nearly any inspection task put in front of them. The one thing they can't do, however, is read the minds of humans. For this reason, industrial automation solution providers need to tell machine vision software what to look for by pre-programming it with specific patterns and logical proceedings.

An Applied Science with Concrete Benefits

Compared with human inspectors, machine vision offers vast improvements in speed and consistency. It also works tirelessly around the clock to consistently deliver quality products and reliably catch botched items. Downtime is minimized as machine vision systems take care of inspection-related tasks quickly, accurately, and consistently, allowing corrective measures to be performed on the fly. Data gathered over time on the suitability of inspected items can also be analyzed to discover trends such as the wear and tear of cutting tools.

One good example of the process control benefits of machine vision is code verification. Essential for the trace-ability of products and product parts, machine-readable codes are printed, dot-peened, or engraved onto components about to embark on the production journey. The quality of codes can be inconsistent and can degrade over time based on the performance of the marking system, so machine vision technology is used to verify the readability of these codes. Advanced code verification systems grade a variety of symbologies according to standards set by the International Organization for Standardization (ISO). They also keep track of grading trends and record information about barcode quality in order to keep an eye on printer functionality and provide a reference in case disputes arise over symbols that don't scan.

Figure 3: Screenshot showing OCV data.

Machine vision can also recognize and verify human-readable symbols — tasks known as Optical Character Recognition (OCR) and Optical Character Verification (OCV). These are important functions because many items bear human-readable codes alongside the machine-readable ones, and both types of marks need to be read to verify that the data matches. OCR is used to do the actual reading, and its strength is in reading characters even when they are deformed. OCV, on the other hand, is used to look at the quality of the printing. It ensures that characters such as a 3, 8, 6, or B are sufficiently clear that they will not be confused when read by a human. Preforming OCV on printed expiration dates is the perfect example of the usefulness of this technology. You would not want anyone misreading an expiration date and ingesting expired food or medicine. Figure 3 shows how OCV looks at the shape of characters on a pixel-by-pixel basis.

Today’s automobiles are more complex and have more selectable options and features than ever before, making production errors increasingly likely. Machine vision helps manufacturers and suppliers prevent defects throughout the production process. It can guide robots to pick parts out of racks, verify that they are the correct parts, inspect the parts for quality, and then position the parts for assembly.

The introduction of smart cameras has greatly improved the options for using machine vision in the automotive industry because it allows for a distributed system of independent inspection at multiple points along the production line. Smart cameras combine optics, lighting, vision software, and communications in a single device, making it unnecessary to have a single processing center for all inspection points.

Capitalizing upon its ability to locate patterns, machine vision technology excels at guiding robotic equipment to pick up necessary objects. Without machine vision, these objects would need to be placed in the exact same spot every time, and the slightest deviation from the expected location would discontinue processing for at least one product and possibly stop the production line altogether. It would also be extremely tricky to direct robots to grasp objects on a moving conveyor belt.

With machine vision, however, finding the location of objects — including moving targets — is relatively easy. The system triggers vision at a specific instant in time, and the software locates the desired objects in the image. The system calculates the position and orientation data for each part, adding what it knows about how far the objects would have traveled from the time it was triggered, and then sends the “pick” location of these parts to a robot. Basically, the computer version of skeet-shooting — where you need to aim and fire the gripper towards a point where the object will be — is what allows a vision-controlled robot to pick up objects on the fly. Given that robot guidance decisions need to be made in a fraction of a second, it's important that the machine vision system, or smart camera, can achieve a near-real-time response.

A Rapidly Evolving Technology

Machine vision is getting more advanced all the time. Once an expensive investment that required a significant amount of programming know-how to implement, this technology is now simple to set up, simple to use, and much, much cheaper than before. The dramatic improvements in accuracy, speed, and reliability have made machine vision — and particularly smart cameras — an extremely hot engineering topic. Given the wide variety of tasks that machine vision systems can already accomplish, it will be exciting to see what innovations the near future brings.

This article was contributed by Omron Microscan, Renton, WA. For more information, Click Here.