From apples to oranges, from avocado to zucchini — the fruit and vegetable departments of large supermarkets offer an overwhelming choice of fresh products. Consumers benefit from this broad selection, as most fruits are now available year round. Another advantage of self-service supermarkets is that consumers can check and choose the fruit and vegetables themselves. Difficulties start, however, when you reach the self-service scale — how to find the right button among dozens of different types of produce. The vast choice can quickly turn into an ordeal when it takes ages to locate the papaya or peppers button. But there might finally be a solution. A compact, integrated USB camera helps self-service scales automatically recognize individual products.
METTLER TOLEDO has been developing and producing weighing systems for food retailers since 1975. The German company’s PC-based UC3 scale series with touch screen is very successful in this market. The series features 20 different models, of which several tens of thousands are in use throughout Europe. The UC3-GTT-P is a very special version: This self-service scale is optionally available with an integrated USB camera from the German machine vision specialist IDS Imaging Development Systems. Only about 3.5 cm in size, the board-level version of the USB uEye LE camera is hidden inside a curved metal arm that is mounted to the scale’s screen. This way, the camera points directly at the weighing plate on which the products are placed, but is barely noticeable to shoppers. The machine vision system has been developed for the shoppers’ convenience and makes the self-service scales exceedingly easy to use.
The analysis process starts automatically as soon as a weight is placed on the scale. The scale’s operating panel displays: “Recognition in progress.” First, the scale checks whether the image captured by the camera changes; for example, because the user’s hand was in the field of view. As soon as the image remains still, the system starts analyzing. This takes about a second. Then four possible matches are presented for selection in large, colored fields.
The consumer can now choose the desired type, and the scale prints the label. Due to various disturbance factors, a fully automatic system cannot be implemented, as highly fluctuating light conditions or covered areas in the image affect the analysis. If conditions are good, however, the camera system achieves a high accuracy even if the weighed products are in plastic bags.
How does the image analysis work? The color is one of the key criteria. The colors detected in the product on the scale are compared with a stored list of colors that are assigned to the individual types of fruit and vegetable. The system analyzes not just single colors, but also color combinations. This way, for example, a specific combination of red, green and yellow could be recognized as “mixed peppers,” if the retailer provides for such a choice. The combined analysis also helps distinguishing between products of similar color, such as lemons and bananas. Characteristics like the shape and texture are additionally evaluated. However, the use of plastic bags often complicates analyzing them. METTLER TOLEDO developed the algorithms in cooperation with the Fraunhofer Institute for Information and Data Processing. “The greatest challenge in developing the system was the actual definition and weighting of the product characteristics,” says product manager Klaus Weber.
When selecting the camera, the size, the USB 2.0 connection and the color quality were at the top of the requirements list. Walter Grom, the engineer in charge of the project, explains: “An important factor in our decision was of course an excellent image quality, which the uEye provides.” The engineers from METTLER TOLEDO opted for the UI-1226LE-C, a model from the USB uEye LE camera series from IDS. The lightsensitive CMOS color sensor of the board-level camera captures up to 87 frames per second at the full 752 x 480 pixel resolution. Another major aspect is that the uEye’s automatic white balance works reliably even against the blue background of the scale.
The second essential factor was the camera’s software integration. Depending on the customer’s specifications, METTLER TOLEDO installs either a Linux-based or a Windows-based operating system on the scale’s integrated PC. “Another reason for choosing IDS was that they also offer a Linux driver that comes right with the camera,” adds Grom. The fact that the machine vision specialist IDS is located not far away from METTLER TOLEDO was a further advantage: During the development stage, the engineer visited IDS a couple of times to inspect the quality of production on-site and discuss various customizations of the camera hardware.
In Germany, the successful combination of precision scale and industrial camera is already in use in over 300 stores of a major food retailing chain. Other retailers in Germany and Europe have also started using the system and are planning to expand usage companywide, as well.