Artificial Intelligence in Design Engineering

In the age of industry digital transformation, there is a data type that has received minimal focus yet can have significant impacts on a company’s success — visual data.

About 80 percent of all impressions from our five sensory organs are from sight alone, meaning the majority of our brain’s decision-making is a result of what we see. Visual data is the easiest, most abundant type of data we can generate using various types of imaging technologies, but it continues to be underutilized.

Computer Vision with artificial intelligence (AI) is capturing this abundant visual data and making sense of it for design engineers, quality teams, production teams, and more. Understanding Computer Vision AI (CV AI) technology is the first step to recognizing how it can work for you.

Computer Vision AI

CV AI is a computer processing and making sense of visual data. Its capabilities can detect and classify different types of defects, people, objects, actions, and events. CV AI learns from the visual data collected to provide predictions and actionable insight on images, videos, and live streams.

A benefit to CV AI is that it allows for changes and updates similar to the way people process things. It can be given feedback to improve the confidence in the predictions, but if a new variable needs to be detected, the computer vision models can be retrained based on new data, new variables, and feedback.

CV AI is camera agnostic, meaning it can be used with any type of imaging technology. It also leverages the latest in deep-learning technology, which is based on artificial neural networks such as convolutional neural networks (CNN) and recurrent neural networks (RNN).

The way CNN work is to provide visual data in the form of images and videos, which are images at some frame rate. The visual data consists of pixels. The pixels are filtered through many layers when it is inputted and through these layers’ features, such as edges, colors, and combinations of those features, are extracted by performing calculations and manipulations. These layers of filters perform convolution operations to detect patterns in the image. Finally, an output layer makes the prediction, and that result is a detection or classification. The object detected is identified and localized in the frame.

Designs of the Future

Design engineers of advanced machines and systems such as robotics, drones, inspections systems, manufacturing lines, and brand-new factories are embedding various types of 2D and 3D imaging technologies along with CV.

Robots and drones of the future need to be able to perceive their environments for more advanced maneuvering and manipulation of objects. To do these, the latest imaging technologies provide accurate visual data to the CV system, which is then making sense of that visual data by detecting and classifying what the images and video are showing.

Instead of an autonomous robot in route stopping because an object is in proximity, the robot is able to understand more precisely what is in its path and determine if it can go around it. In a case where a robot detects a box in its path, it can then confidently adjust its path plan to go around the stationary box, virtually losing no time. In more advanced use cases, drones are used to inspect surfaces of critical assets for damage or defects. CV is used to detect and classify defects and provide further analytics on what defects were found and where.

The factory of the future is connected, intelligent, and needs to be able to ensure highly productive production environments. One of the key enablers is CV. The technology is empowering AR tools, advanced quality inspections, smarter robots, and intelligent safety and security systems. Design engineers are integrating cameras into machine designs and production lines at the design phase.

Quality and production effectiveness is ensured when intelligent tools are integrated in critical areas. For instance, in assembly and joining processes where two major parts are joined in a welding process, cameras can monitor the process and CV can determine the weld quality. In an instance where porosity is detected, the welding engineer for the production line or even a machine itself can make adjustments to the process in near real-time. By doing so productivity and quality is improved and risks are mitigated.

A Unique Imaging Solution

CV AI is much more powerful than smart AI image sensors or rules-based vision as a whole. Usually “smart” vision sensors leverage pre-trained models that are not retrainable from the perspective of the camera. Even “edge” training smart AI sensors are highly limited to the most basic applications. Some sensors or AI systems rely on open-source tools that are trained on datasets that could be subject to biases or inconsistent labels.

Rules-based vision uses decades old rules that are based on the specific object, lighting, camera, lens, and ambient conditions. These rules are good for simple detection applications but fail when contending with detecting things in changing or challenging backgrounds, shapes, colors, sizes, lighting, and environments.

Because CV AI allows for changes and updates, it can evolve with a user’s needs. For example, if new variables need to be detected, the computer vision models can be retrained based on new data, new variables, and feedback. It can also learn to ignore variables or conditions that change but do not matter when making detections and give feedback to improve the confidence in the predictions.

CV AI can also eliminate the risk of valuable knowledge leaving with a person because it learns and retains visual inspection requirements in complex production environments. Retaining that tribal knowledge is also impactful when onboarding new employees, as it reduces weeks and even perhaps months of training time.

One of the most appealing attributes of CV AI is that it is camera agnostic and can work with any type of imaging technologies on the market, including camera technologies such as resolution imager, visible light, invisible (NIR, SWIR), X-ray, thermal, industrial cameras, CCTV Cameras, microscopes, and borescopes.

Understanding How CV AI Can Impact You

To understand if CV AI could be beneficial to your design engineering process, you should ask yourself these questions:

• Are the challenges faced in your design engineering process visible by some form of camera technology or any spectrum that provides image or video?

• What aspects of your operations can image or video detections and analytics bring value?

• What insights do you want to gain from CV AI?

• What systems will CV AI integrate into so that action can be taken because of detections and analytics, i.e., machine, MES, or ERP?

• Is there a one-off need, or can this span across the operations?

• Does the platform empower your SMEs to easily build their own detectors without needing to be data scientists?

By looking at these areas of your manufacturing process, you can gauge how and where CV AI will be most beneficial.

What’s Next

If you’ve decided that your design engineering process can benefit from CV AI, there are simple steps you can take to do to get started with this technology and ensure that it’s a good fit for your organization:

• Consult with a CV AI platform company that can review your interests, applications, and goals for using CV AI.

• Consider a PoC trial to learn how the technology can help and integrate into your systems and operations.

• Ensure that it compliments your operations by allowing your cross-functional departments to build, test, deploy, and manage pre-made and custom detectors for their needs without needing to be a degreed data scientist.

• Ensure suitable deployment modes for your needs, such as Cloud (public/private), On-prem, and edge.

• Confirm that the CV AI platform is camera technology agnostic so that it can work with any technology. Application needs vary throughout operations and different imaging technologies and resolutions may be needed.

• Use an application that can integrate in your current systems, i.e., PLC’s, SCADA, MES, WMS, ERP, Databases, Security, etc.

• Partner with proven talent with a track of deep-learning AI. Industry-leading engineering is what is empowering the greatest gains for manufacturers utilizing this technology.

Industry digital transformation is making its way into all aspects of operations from leveraging tools that have these technologies incorporated into them, to incorporating the technologies into new designs. Manufacturing is expected to grow rapidly across many regions. The competitive advantage will be for those that incorporate technologies which maximize throughput and quality. CV AI is one of the key technologies that will be incorporated into new designs of robotics, machines, and production lines within the factories of today and tomorrow. This will enable robotics and machines with advanced visual perception to do more faster and adapt to changing conditions and environments.

This article was written by Adam Bennett, Sales Director, Matroid (Palo Alto, CA). For more information, visit here  .

---