“Generally speaking, there are two major shifts that can occur with sensors, and therefore, camera design,” Trisha Hensley, executive assistant at Edmund Optics told me. “Pixels get smaller or sensors get larger. We then tend to see this cycle repeat, as both of these changes improve camera resolution in their own ways.”
Right now, sensors are getting larger, delighting photographers with their ability to gain more information, exclude noise, and import more light than their smaller-sensored forebears.
Sensor development affects lens manufacturing, since larger sensors need larger lenses. According to Hensley, “While C-mount lenses are still quite relevant and ubiquitous in machine vision, we are starting to see newer sensors push these lenses beyond their physical limits.” (C-mount lenses are commonly found on 16 mm movie cameras, closed-circuit television cameras, and machine vision cameras.)
“On the whole,” Sam Sadoulet, President and Chief Operating Officer at Edmund Optics said, “the quality and performance of the lens within these systems will become even more important than it is now.”
Sensors Evolve to Meet Expanding Machine Vision Needs
Over the last 12 – 18 months, products from digital single-lens reflex (DSLR) cameras to smartphones pushed the standard pixel range to a maximum of 5 μm to 9 μm and beyond — generally capping at 20 μm.
In the smartphone market, Nokia leads the way with its 808 PureView, which packs a 41-megapixel image sensor. The 808 condenses seven pixels into one, magnifying photographs and making them sharper.
Sony’s RX100 has energized the market for compact cameras — a niche that struggles to survive among smartphones and mirrorless or DSLR cameras. The RX100 crams 20.2 megapixels on a one-inch-type sensor, producing exceptional quality images for its compact size.
Other compact point-and-shoot cameras like Canon’s PowerShot G-series, with its 1.5-inch, 14.3-megapixel High-Sensitivity Canon CMOS sensor — six times bigger than those in previous G-series models, and Fujifilm’s X100, with its large APS-C sensor and precisely matched prime lens, give the cameras more ISO sensitivity and sharper images. Of its X100, Fujifilm says that “The combination of a hybrid viewfinder, large APS-C sensor and a precisely matched prime lens, makes this camera unique compared to anything else in the market.”
Larger sensors mean more expensive devices, so DSLR cameras, like the Nikon D600 and Canon EOS 6D compete by making big sensor shooting affordable.
Three of the Most Innovative Image Sensors are Toshiba, Samsung Zero and Sony
As Quenton Hall, AI Systems Architect of Xilinx, told participants at the May 2020 AIA Vision Week conference for machine vision education, one of the biggest concerns is the amount of energy that sensors consume. “Currently, there are tens of millions of IP cameras that are installed worldwide annually. Not all are connected to the internet, but if you approximate [even a fraction of this] they consume a huge amount of internet bandwidth, estimated by Cisco to increase by 2021 to approximately 3.4 percent of all internet traffic.” Hall continued: “We need to find a way to squeeze these powerful algorithms into low-power, low-cost devices… We need to batch images for peak efficiency to reduce memory churn and pipeline stall.”
Toshiba’s TCD1105GFG and TCD1106GFG sensors, with their maximum data rate of 25 MHz and single 3.3 V power supply, are energy-efficient, with unprecedented high-speed performance.
In the smartphone camera industry, Samsung’s 50MP ISOCELL GN1 sensor produces high-quality images in low-light conditions through an innovative method that not only isolates pixels for more light but also expands pixel size from the standard 0.8 μm of competitors to 1.2 μm.
Sony revolutionized the industry with its IMX990 and IMX991 SWIR sensors. Specifically, Sony’s new image sensors can sense short-wavelength infrared (SWIR) light as well as visible light. This makes these sensors particularly valuable for deep learning and artificial intelligence tasks, where the cameras can pick up almost invisible defects in manufacturing parts that standard sensors overlook.
“This is the biggest technology leap I’ve seen in sensors in recent history,” Steve Kinney, Director of Engineering at Smart Vision Lights, told me, “In the near term, these SWIR sensors will rapidly change the game for machine vision-based applications that would benefit from SWIR imaging. These include medical research, materials research, and material handling applications, some of which might be driven by the COVID crisis.”
“For example,” he added, “SWIR imaging can allow materials, like certain adhesives, to be seen through other materials. Some areas like personal protective equipment (PPE) are manufactured with a high-speed web process, where multiple thin layers are bonded or edge-glued together. The benefits of SWIR imaging afforded by the new Sony IMX 990/991 sensors may greatly help this production as the demand for PPE rapidly increases due to COVID.”
Covid’s Impact on the Camera Industry
“I think that overall, COVID-19 hasn’t done much to halt the progress or significantly alter the trajectory of the trends and where designs are headed,” Sadoulet told me, “it has merely slowed it down, especially in Europe and the Americas.”
“That’s because,” he added, “many of the ways that imaging shapes the future are forward-looking, long-term projects, so the pandemic has not significantly derailed them.”
In other words, since the camera industry is driven by sensor trends whose impact stretches into the future, the industry remains scarcely affected by the relatively short COVID situation.
All the same, the camera industry suffered. Major trade shows, like NAB, were cancelled. New products were delayed as factories and businesses closed. All major camera companies transferred their work off-base. Sony had to close some manufacturing plants in Malaysia; Nikon delayed its anticipated D6 DSLR; Ricoh and Canon suspended the operation of some of their production lines in Japan; Sigma closed its U.S. offices; and Fujifilm’s popular X-T30 and X-T3 models stalled, while its new X-T4 was being held in limbo.
All camera companies are operating at reduced capacity and since most companies source their manufacturing from China or Japan, they’re also held back by supply constraints. However, contrary to manufacturers’ expectations, none of that has affected sales trends.
In fact, there has been an uptick in sales of hyperspectral, multispectral, and automated high magnification inspection systems, needed to check the quality of Personal Protective Equipment (PPE) like surgical masks, gloves, and gowns.
“In my opinion,” Kinney told me, “sales of multispectral and hyperspectral cameras are dwarfed by the large uptick in standard camera equipment that is indirectly attributed to COVID due to their use in logistics systems for automated distribution systems and smart warehouses.”
Eric Danzinger, CEO of Invisible AI, told participants at the AIA Vision Week conference: “Our world’s changed dramatically. Our operating environment is different. We can’t rely on face-to-face interactions to solve problems on the factory floor. We can’t meet in person. We can’t look over someone’s shoulder to supervise their work. This creates a lot of stress.”
On the other hand, the Covid crisis creates opportunity. “Imaging technology helps us have automated warehouses that are able to handle higher volumes of goods, while simultaneously keeping their employees safe,” Sadoulet told me.
Companies like Instrumental innovated smart camera devices that could discover and fix production issues remotely. Engineering, quality, and operations leaders use these mobile devices to review how their products are being assembled, find errors, and make changes.
“Devices such as these replace face-to-face factory floor interaction with real time digital surveillance. And so now you’re seeing a big uptick in production,” Danzinger said. “So instead of having folks on the ground, they can get all this data uploaded to the cloud and automatically see where they are having problems or where a part may have failed. So being able to capture video and catalogue it — not just by time but also by part or member — provides a huge advantage.”
“Cameras,” he continued, “help us get data from every single machine… generate data from the actions of employees… make this living digital copy of our facility that allows us to see problems throughout time and to simulate improvements, all without having to do manual time studies or constantly walk the floor. In that way, we’re moving to the Factory of the Future — doing things like in retail and in telemedicine — where these industries achieve years of progress through digitization in just a matter of weeks.”
Where is the Camera Industry Headed?
“With vision and imaging pushing into more and more markets, such as autonomous systems,” Hensley said, “I would expect to see more multi- and hyperspectral cameras getting developed, as well as seeing another round of pixels getting smaller. All of these new requirements will put additional stress on the optics within these next generation systems, and the quality and performance of the lens within these systems will become even more important than it is even now.”
Those long-term projects for a post-Covid world, will be possible because the camera industry is using its current challenges for disruptive transformation.