Camera Inspired by the Human Eye Can Improve How Robots See, React

University of Maryland, College Park, MD

A demonstration of how microsaccades counteract visual fading. After a few seconds of fixation (staring) on the red spot in this static image, the background details of this image begin to visually fade. This is because microsaccades have been suppressed during this time and the eye cannot provide effective visual stimulation to prevent peripheral fading. (Image: UMIACS Computer Vision Laboratory)

A team led by University of Maryland computer scientists invented a camera mechanism that improves how robots see and react to the world around them. Inspired by how the human eye works, their innovative camera system mimics the tiny involuntary movements used by the eye to maintain clear and stable vision over time. The team’s prototyping and testing of the camera — called the Artificial Microsaccade-Enhanced Event Camera (AMI-EV) — was detailed in a paper published in the journal Science Robotics in May 2024.

“Event cameras are a relatively new technology better at tracking moving objects than traditional cameras, but today’s event cameras struggle to capture sharp, blur-free images when there’s a lot of motion involved,” said Lead Author Botao He, Computer Science Ph.D. student at UMD. “It’s a big problem because robots and many other technologies — such as self-driving cars — rely on accurate and timely images to react correctly to a changing environment. So, we asked ourselves: How do humans and animals make sure their vision stays focused on a moving object?”

For He’s team, the answer was microsaccades, small and quick eye movements that involuntarily occur when a person tries to focus their view. Through these minute yet continuous movements, the human eye can keep focus on an object and its visual textures — such as color, depth and shadowing — accurately over time.

The team successfully replicated microsaccades by inserting a rotating prism inside the AMI-EV to redirect light beams captured by the lens. The continuous rotational movement of the prism simulated the movements naturally occurring within a human eye, allowing the camera to stabilize the textures of a recorded object just as a human would. The team then developed software to compensate for the prism’s movement within the AMI-EV to consolidate stable images from the shifting lights.

“Our eyes take pictures of the world around us and those pictures are sent to our brain, where the images are analyzed. Perception happens through that process and that’s how we understand the world,” explained Co-Author Yiannis Aloimonos, Professor of Computer Science at UMD, who is also Director of the Computer Vision Laboratory at the University of Maryland Institute for Advanced Computer Studies (UMIACS). “When you’re working with robots, replace the eyes with a camera and the brain with a computer. Better cameras mean better perception and reactions for robots.”

The researchers also believe that their innovation could have significant implications beyond robotics and national defense. Scientists working in industries that rely on accurate image capture and shape detection are constantly looking for ways to improve their cameras — and AMI-EV could be the key solution to many of the problems they face.

“With their unique features, event sensors and AMI-EV are poised to take center stage in the realm of smart wearables,” said research scientist Cornelia Fermüller, senior author of the paper. “They have distinct advantages over classical cameras — such as superior performance in extreme lighting conditions, low latency and low power consumption. These features are ideal for virtual reality applications, for example, where a seamless experience and the rapid computations of head and body movements are necessary.”

For more information, contact Georgia Jiang at This email address is being protected from spambots. You need JavaScript enabled to view it.; 301-405-5204.