Using hardware similar to what enables autonomous cars to “see” the world around them, researchers have developed a system that can see through clouds and fog. They enhanced the system with an algorithm that can reconstruct three-dimensional hidden scenes based on the movement of individual particles of light, or photons. In tests, the system successfully reconstructed shapes obscured by 1"-thick foam. To the human eye, it’s like seeing through walls.

This technique complements other vision systems that can see through barriers on the microscopic scale — for applications in medicine — because it’s more focused on large-scale situations such as navigating self-driving cars in fog or heavy rain and satellite imaging of the surface of Earth and other planets through hazy atmosphere.

In order to see through environments that scatter light, the system pairs a laser with a super-sensitive photon detector that records every bit of laser light that hits it. As the laser scans an obstruction like a wall of foam, an occasional photon will manage to pass through the foam, hit the objects hidden behind it, and pass back through the foam to reach the detector. The algorithm-supported software then uses those few photons — and information about where and when they hit the detector — to reconstruct the hidden objects in 3D.

This is not the first system with the ability to reveal hidden objects through scattering environments but it circumvents limitations associated with other techniques; for example, some require knowledge about how far away the object of interest is. It is also common that these systems only use information from ballistic photons, which are photons that travel to and from the hidden object through the scattering field but without actually scattering along the way.

In order to make their algorithm amenable to the complexities of scattering, the researchers had to closely co-design their hardware and software, although the hardware components they used are only slightly more advanced than what is currently found in autonomous cars. Depending on the brightness of the hidden objects, scanning in their tests took anywhere from one minute to one hour but the algorithm reconstructed the obscured scene in real time and could be run on a laptop.

Someday, a descendant of this system could be sent through space to other planets and moons to help see through icy clouds to deeper layers and surfaces. In the nearer term, the researchers would like to experiment with different scattering environments to simulate other circumstances where this technology could be useful.

For more information, contact Taylor Kubota at This email address is being protected from spambots. You need JavaScript enabled to view it.; 650-724-7707.