Compound Eye Structure Could Revolutionize Robotic Vision

Andrew Corselli

A research team at the School of Engineering of the Hong Kong University of Science and Technology (HKUST) recently developed a novel artificial compound eye system that is not only more cost-effective but demonstrates a sensitivity at least twice that of existing market products in small areas. The system promises to revolutionize robotic vision, enhance robots' abilities in navigation, perception, and decision-making, while promoting commercial application and further development in human-robot collaboration.

Mimicking the visual capabilities of compound eyes, this innovative system can be applied in a wide range of scenarios, such as installing on drones to improve their accuracy and efficiency in tasks like irrigation or emergency rescue in disaster sites. With its high sensitivity, the system can also enable closer collaboration among robots and other connected devices. In the long term, the compound eye system will enhance autonomous driving safety and accelerate the adoption of intelligent transport systems, fostering the development of smart cities.

Traditionally, roboticists have mainly focused on replicating the visual capabilities of insects, which offer a wide field of view and advanced motion-tracking capabilities. However, integrating compound eye systems into autonomous platforms like robots or drones has been challenging, as these systems often suffer from issues related to complexity and stability during deformation, geometry constraints, as well as potential mismatches between optical and detector components.

To address these challenges, Professor Fan Zhiyong’s team developed a pinhole compound vision system by adopting new materials and structures. This system features several key characteristics, including an inherent hemispherical perovskite nanowire array imager with high pixel density to enlarge the imaging field; and a 3D-printed lens-free pinhole array with a customizable layout to regulate incident light and eliminate the blind area between neighboring ommatidia (individual units within an insect’s compound eye). Owing to its good angular selectivity, a wide field of view, wide spectrum response in monocular and binocular configurations, as well as its dynamic motion tracking capability, the pinhole compound eye not only can accurately locate targets but can also track a moving quadruped robot after incorporated onto a drone.

“This compound eye design is simple, light, and cheap,” Fan said. “Although it won’t fully replace traditional cameras, it could be a huge boost in certain robotics applications, such as in a swarm of drones flying in close formation. By further miniaturizing the device size and increasing the number of ommatidia, imaging resolution, and response speed, this type of device can find broad applications in optoelectronics and robotics.”

Here is an exclusive Tech Briefs interview with Fan, edited for length and clarity.

Tech Briefs: What was the biggest technical challenge you faced while developing this compound eye system?

Fan: This ultra-thin hemispherical photo detector array, we call it an artificial retina. It's just like the human retina we have in our eyes. It's a very, very thin structure. The diameter is around 2 cm, it's like an eggshell. Everything is a kind of fragile. So, the team laminated the structure from the aluminum substrate.

We have this thin membrane protected by the hard eggshell. So, if we wanted to build off the membrane from the inside of the action, it's very challenging. It's a very delicate job. That was an engineering challenge over there.

Tech Briefs: How did this project come about? What was the catalyst for the work?

Fan: Our research group started to work on the bionic eye project in 2016. In 2020, we published a paper to report the single eye, like a human eye. But Mother Nature created two kinds of separate eyes: One is our single eye, like ours or mammals’, the other type of eye, called compound eyes, is for insects because they have a very small brain and very limited computation power. So, their eye structure cannot be very sophisticated. Mother Nature developed this so-called compound eyes structure with many, many small eyes. We have been very interested to replicate those Mother Nature-developed structures.

These are the advanced version of photo detectors — single eye or compound eye. Since we already made a single eye, after 2020 we were thinking whether we could make the compound eye. And then we did it. The two eyes are different, and not only on the structure, but the functionality is also different. Like bees, for example. Normally, they come in a swarm, maybe thousands of them. They have a very interesting function in their eyes. It is an optical avoidance, a collision avoidance, because their compound eyes are very sensitive to the motion for moving objects.

If their workmates are moving around them, then they know the precise location, the velocity, etc. That is a very interesting function. We're interested in this kind of function because that's important for many applications, such as drones, autonomous vehicles, robotics. Although these compound eyes do not really give you a high-resolution image like a single eye, they have unique merits that actually triggered our research.

Tech Briefs: What are your next steps?

Fan: We are still developing this compound eye structure — particularly, improving our pixel density. Right now, we only have a couple hundred sensing pixels, still far away from real insects — they have thousands. One direction we're working on is improving the number of pixels. We need to shrink the size of each pixel, and we need to build an optical structure on top of each pixel. This is not an easy job, but we are working on it. We developed some new fabrication methods; we can go beyond 100,000 pixels already. That is already more than some insects, but we still need to develop the optical structure and align optical structure on top of each pixel.

Also, we’re very interested in exploring the applications. In our work, we have demonstrated applications — we put the compound eye on a drone, and the drone is able to track the motion of a robotic dog on the ground. The dog is carrying a light source, and the drone is looking at the light source position and trying to follow the light source. And we are interested to explore more applications for drones and autonomous vehicles in the future, but that requires a lot more work.

[In addition to the aforementioned hindrances], we also need to develop some scalable fabrication technology to reduce the cost of a fabrication, if you want to make it for practical applications or causes.

---