Cameras are the eyes of the modern world. The devices equip today’s cars and machines, and real-time imaging capabilities support law enforcement, businesses, and homeowners. Thus far, cameras have been just that: simple cameras, eyes without a brain. Their sole purpose has been, primarily, to output images. Nowadays, however, cameras are “smarter.” In this article, we will review how imaging technology’s ability to intelligently process data will support new insights and applications.

A camera detects a cardboard box. For security and surveillance purposes, object detection capabilities can be performed in specific zones, such as airports, trains, and bus stations. (Image Credit: IntelliVision Technologies)

‘Intelligence’

What makes a camera intelligent is its ability to understand and interpret images or videos, and to take action with that information. A smart camera can send event alerts, track objects, and intelligently search through volumes of visual data. In order to achieve such capabilities, the CPU or GPU inside the camera must constantly receive video and process it simultaneously — to understand it. When a camera has the ability to take action, to send alerts, or convey information or metadata in a meaningful way to the user, this is called actionable intelligence.

In an effort to process real-time video and information at the very edge of a connected IP network, the camera has become the focal point for new innovation and development. Just ten years ago, the camera was simply a device for recording video or producing images. Today, the camera has become highly intelligent, mainly through the advent of advances in semiconductors, software, and math algorithms.

The IP cameras are not only accessible over the Internet, but now they also represent a visual edge of a communications network. Having the camera “sit at the edge” goes beyond simple connections between devices; today’s devices are connected in an intelligent network. With the addition of powerful and cost-effective computers embedded into smart cameras, computer technology has been pushed to the edge, or into the camera itself.

Today’s cameras can recognize individual faces, as shown in the above image. (Image Credit: IntelliVision Technologies)

Emerging Application Areas

While smart cameras can be used in a way that benefits a variety of applications or industries, here are three emerging areas:

  • Smart Home
  • Smart Retail
  • Smart Cars

For a home device to detect intruders, for a retailer to spot customers, and for a car to find pedestrians, the camera must first understand the background of a scene and then continuously look for changes, or objects in motion. When a valid change is detected, the camera's software and algorithm must perform classification. For example, is the shape a person? A vehicle? An animal?

Finally, “recognition” intelligence allows cameras to “learn” faces or objects, often through a user supplying his or her own data to “train” the system. Based on a database of “learned” faces, the latest artificial intelligence, or deep learning, software can then recognize a detected individual as a son, daughter, or spouse.

Heat maps display customer traffic and areas with the greatest amount of activity. Such "hot zones" are shown above in red and yellow. (Image Credit: IntelliVision Technologies)

Smart Home Cameras

Home cameras are becoming increasingly more common. Featuring built-in intelligence, the small, Wi-Fi-enabled devices can be used for indoor, outdoor, doorbell, or roadside use. The “brain” inside the camera can look for people, intelligently detect motion, spot a human entering a given zone, track a passing car, find an individual’s face, or even recognize the license plate of a car and check it against a watch list.

All of this smart processing can be done inside the camera. Soon, smart cameras will be able to send parents a notification when their child arrives home, when a stranger or a parcel delivery appears at a doorstep, or when someone has been loitering around the property. This processing is all done instantly, within the camera’s internal CPU, by video content analysis; the output, or alert, is then passed directly to the user.

The advancements in video content analytics have now made it possible to monitor many situations in real time, and to take action accordingly. In this market, the camera has become one of the primary sensors in the home. With such a connected environment, the IoT, or Internet of Things, cloud is now an ideal place to extend camera intelligence.

Smart Retail Stores

Additionally, many “smart” retail outlets are taking full advantage of smart cameras and cloud-based services. The technology not only provides security, surveillance, and peace of mind to store owners, but the data generated by the smart cameras also creates important insights for businesses.

The intelligent cameras perform many retail data extractions, such as customer counts, queue lengths, report service times at the teller, and customer interest in specific sales and/or products.

A dashboard displays a retail store's consumer data, including percentage of male/female customers, age group, and time spent shopping. (Image Credit: IntelliVision Technologies)

Business intelligence can be gathered to monitor point-of-sale (PoS) activity, queue length/time and duration, customer demographics, or traffic maps, showing in graphical format where high-volume traffic is concentrated in a store. Such business intelligence can ultimately be used to directly drive productivity and profits in stores.

Smart Cars

Your new car probably already has multiple cameras. The devices are part of a vehicle’s Automated Driver Assistance Systems (ADAS), which send critical information to operators behind the wheel: lane change warnings, front collision alerts, pedestrian detection, and reverse/back-up collision signals, for example. Many vehicles use smart cameras to provide alerts and actionable intelligence to the driver, thus saving lives in many areas.

Many cars today have more than one camera; some luxury models have up to nine imaging devices. The new “eyes” of the vehicle are not only recording video; they are also using the data to physically initiate a car’s actions — an essential capability as fully autonomous vehicles appear on the road.