A real-time 3D motion tracking system combines transparent light detectors with advanced neural network methods to create a system that could one day replace LiDAR and cameras in autonomous technologies. The imaging system exploits the advantages of transparent, nanoscale, highly sensitive graphene photodetectors.
The graphene photodetectors were tweaked to absorb only about 10% of the light they’re exposed to, making them nearly transparent. Because graphene is so sensitive to light, this is sufficient to generate images that can be reconstructed through computational imaging. The photodetectors are stacked behind each other, resulting in a compact system, and each layer focuses on a different focal plane, which enables 3D imaging.
The team also tackled real-time motion tracking, which is critical to a wide array of autonomous robotic applications. To do this, they needed a way to determine the position and orientation of an object being tracked. Typical approaches involve LiDAR systems and light-field cameras, both of which suffer from significant limitations. Others use metamaterials or multiple cameras. Hardware alone was not enough to produce the desired results.
The team built an optical setup and enabled a neural network to decipher the positional information. The neural network is trained to search for specific objects in the entire scene and then focus only on the object of interest; for example, a pedestrian in traffic or an object moving into your lane on a highway. The technology works particularly well for stable systems, such as automated manufacturing, or projecting human body structures in 3D for the medical community.
The type of algorithms the team developed are unlike traditional signal processing algorithms used for long-standing imaging technologies such as X-ray and MRI. The team demonstrated success tracking a beam of light as well as an actual ladybug with a stack of two 4 × 4 (16-pixel) graphene photodetector arrays. They also proved that their technique is scalable. It would take as few as 4,000 pixels for some practical applications and 400 × 600-pixel arrays for many more.
While the technology could be used with other materials, additional advantages to graphene are that it doesn’t require artificial illumination and it’s environmentally friendly.