Helping Machine Vision Go Beyond Human Programming

Nearly every company in the world performs some level of quality inspection on their products before delivering them to customers. If you’re in the downloadable software business, this might involve making sure the product is bug-free and easy to use. But in the realm of physical products, the appearance of the product is nearly as important as its functionality. Would you want to purchase a new car that has scratches on the bumper or hubcaps? What if there was a crack in the windshield? From large to small, the same is true of many other items including appliances, laptops, cellphones, watches, and earbuds.

Consumers’ perception of quality also extends to the product packaging. These products could be food items, beverages, pharmaceuticals, non-food grocery items, and much more. Even the packaging of garbage bags could influence a buying decision. Are the labels in the right place? Are they the right color? Is the ink or printing smeared? Is the packaging torn, crushed, or unglued? How often does someone pick up an item in the grocery store to find something wrong with the packaging and they put it back to search for a better one? This behavior may seem silly, but many people do it every day.

Resolving the Quality Inspector’s Challenge with AI

If the product is fully functional, but it doesn’t look right to the customer, it will get passed over. Since there are so many elements to quality inspection, it can sometimes be difficult to define what a good-quality product should look like. This can make it very challenging for quality assurance inspectors, machine maintenance personnel, and the engineers who program inspection systems. It takes time to make the inspection process more objective; lists of product and packaging defects are often created and quantified.

Furthermore, this process requires someone — or maybe even a team of experienced people — who can understand the product and the variety of possible defects. How big does a scratch need to be before the product is scrapped? How much color variation is allowed in the packaging? If the position of various components is slightly off, but the product is still fully functional, is this a defect criterion? The plethora of possible product pitfalls can make the head swim. And the time and personnel investment to do quality inspection correctly is costly to a company especially when they may have many, many different products.

This is where a quality inspection vision system with artificial intelligence (AI) can really make a difference to you, to your company’s business, and to your company’s perception in the marketplace. Artificial intelligence is no longer science fiction, and there’s evidence of it in our everyday lives. It runs on computer servers, in satellites, in our cars, on cellphones, and much, much more. It was created to mimic the human brain, and it will one day exceed it in speed of calculation and depth of knowledge.

As a scientific endeavor, machine learning grew out of the quest for AI. Machine learning is a subset of AI and is data-centric. Machine learning algorithms build a model based on sample data in order to make decisions without being explicitly programmed to do so. Deep learning, which is part of machine learning, distinguishes itself by the type of data it uses and the methods in which it learns. Deep learning is essentially a neural network with three or more layers. Neural Networks attempt to simulate the behavior of the human brain by taking unstructured data and giving it meaning. Data that is correlated to the correct prediction of an outcome is positively weighed, and data that does not affect — or that impedes — the correct prediction of an outcome is negatively weighed or excluded altogether.

Over the last few years, vision systems and smart cameras have started to include AI. Artificial intelligence not only helps to solve the most common applications, but also offers some major benefits by reducing commissioning time and minimizing the level of expertise required for setup and maintenance. Machine vision with AI is capable of capturing defects with human-like sensitivity and beyond “seeing” faster and with greater detail than the human eye. AI validates good products — including their many acceptable variations — with similar precision relative to an experienced human inspector.

Implementing AI-Enhanced Vision Inspections and Building an AI Model

No special engineers or environments are required for an AI implementation. Costly, highly specialized AI software and process control engineers do not have to design and program the hardware and software for these AI inspection systems. They have all the computing hardware they need self-contained in a small vision system controller or smart camera. Multiple pieces of computing hardware with extra RAM, extensive computer networks, data processing servers, and extra image storage locations are no longer required. AI inspections allow a programmer to provide good and bad images of a part. The system then uses the images to build its neural network layers that will analyze future images based on features that vary from product to product. These features include edge locations and borders, blob sizes with or without holes filled, image subtractions from a “golden image” averaged using all good images, regions of interest and omission, average defect size and count, and more.

The AI models built by these inspections do not require a lot of images for good or bad parts. However, it is helpful to have more images available, because the AI will advise the programmer as to which good images are significantly different from other good images and should be included in the next AI model build. Additionally, bad product images may be input by themselves, or the programmer may specify what makes the image bad by highlighting a particular region (which may be almost any shape). After the AI model has been built, it is ready to be tested with more images that were not previously provided to the AI. If the AI model performs to the programmer’s satisfaction, it is now ready for deployment on the actual application.

Another more powerful AI inspection automatically detects scratches on a product. To make it work, all the programmer needs to do is specify a region of interest within the camera’s field of view. This inspection uses multiple algorithms in its AI model to automatically detect scratches on varying types of materials and finishes. Scratches can be detected whether they have high or low contrast, and may have straight, broken, jagged, or curled geometries. This inspection is also flexible enough to be used for other types of applications including looking for foreign materials on products, cracks in glass or ceramics, tears in fabrics, and more. The programmer can take the detected scratch area and run it through additional inspections to check for pass or fail criteria. For example, the scratch image could be used in a blob analysis to determine the length of the scratch’s major axis. If the scratch’s length gets too long, then the image fails, and the bad product is rejected. The maximum acceptable scratch length can be set by the programmer.

What’s Around the Corner for AI-Enhanced Vision?

Automation solutions providers are continuing the development of AI in many different products. New types of AI vision inspections are being added to vision systems and smart cameras. Existing, powerful inspections for optical character recognition, 1D and 2D code reading, facial recognition, and more will be augmented with AI features in the near future. Even with the same hardware, vision systems’ and smart cameras’ firmware and software can be upgraded as developers release new AI inspections. By simply selecting a new AI inspection type within the same system, you will be able to read characters and codes printed on curved surfaces, printed at variable rates of speed which elongate or compress characters and code features, angled or staggered print, and print with poor contrast due to lighting issues or depleting ink. The AI will recognize each individual character, correcting for aspect ratio distortion, orientation, and contrast, and piece together the meaning of the characters or codes. The AI can also gain understanding of what the codes should mean by previously read codes and can indicate a readability grade for the characters or codes.

Facial recognition AI inspections will go beyond simple geometry. They will learn how makeup, tattoos, or clothing may conceal features that were previously used to identify someone. Once these identity-hiding factors are recognized by the AI, they can be compensated for during the inspection. Even measurement inspections will become more reliable with AI features. When lighting and contrast fail, the AI inspections will still use the correct features to make each dimensional measurement.

How can your company make AI work for you? Do you want AI to complete all the quality checks on your products before they ship to your customers? This will help to ensure your customers’ satisfaction — and keep them buying products from you — for many years to come.

This article was written by Tad A. G. Newman, Product Manager for Machine Vision, RFID, and Precision Measurement, Omron Automation Americas. For more information, visit here  .

---