Self-flying drones and autonomous taxis that can safely operate in fog may sound futuristic, but new research at Sandia National Laboratories’ fog facility is bringing the future closer.
Fog can make travel by water, air, and land hazardous when it becomes hard for both people and sensors to detect objects. Researchers at Sandia’s fog facility are addressing that challenge through new optical research in computational imaging and by partnering with NASA researchers working on Advanced Air Mobility, Teledyne FLIR and others to test sensors in customized fog that can be measured and repeatedly produced on demand.
Built in 2014, Sandia’s fog chamber is 180 feet long, 10 feet tall and 10 feet wide. The chamber is lined with plastic sheeting to entrap the fog. When the team begins a test, 64 nozzles hiss as they spray a custom mixture of water and salt. As the spray spreads, the humidity builds and thick fog forms. Soon, an observer inside won’t be able to see the walls, ceiling or entrance through the aerosol and people and objects a few feet away will be obscured or completely hidden.
Sandia’s researchers carefully measure properties of fog over time to understand how it forms and changes. By adjusting environmental parameters, the researchers can change the fog properties to better match naturally occurring fog.
“Our team can measure and completely characterize the fog that we produce at the facility, and we can repeatedly generate similar fog on different days,” said Andres Sanchez, chemical engineer. “Having consistent and measurable conditions is important when we’re testing how sensors perform in fog.”
Researchers from NASA’s Ames Research Center recently visited Sandia to perform a series of experiments to test how commercially available sensors perceive obstacles in fog. The Revolutionary Aviation Mobility group is part of the NASA Transformational Tools and Technologies project.
“We tested perception technologies that might go into autonomous air vehicles,” said Nick Cramer, the lead NASA engineer for this project. “We want to make sure these vehicles are able to operate safely in our airspace. This technology will replace a pilot’s eyes, and we need to be able to do that in all types of weather.”
The team set up a stationary drone in the chamber as a target and then tested various sensors to see how well they could perceive the drone in the fog.
“The fog chamber at Sandia National Laboratories is incredibly important for this test,” Cramer said. “It allows us to really tune in the parameters and look at variations over long distances. We can replicate long distances and various types of fog that are relevant to the aerospace environment.” Cramer said one of the challenges of self-flying technology is that there would be a lot of small vehicles flying in close proximity.
Teledyne FLIR has tested its own infrared cameras at Sandia’s fog facility to determine how well they detect and classify pedestrians and other objects. Chris Posch, engineering director, automotive, for Teledyne FLIR, said the cameras could be used to improve both the safety of today’s vehicles with advanced driver-assisted systems features such as automatic emergency breaking and autonomous vehicles of the future.
“Fog testing is very difficult to do in nature because it is so fleeting and there are many inherent differences typically seen in water droplet sizes, consistency and repeatability of fog or mist,” said Posch. “As the Sandia fog facility can repeatedly create fog with various water content and size, the facility was critical in gathering the test data in a thorough scientific manner.”
Sandia and Teledyne FLIR conducted multiple performance tests with vehicle safety sensors including visible cameras, longwave infrared cameras, midwave infrared cameras, shortwave infrared cameras and lidar. Posch said the results showed that Teledyne FLIR’s longwave infrared cameras can accurately detect and classify pedestrians and other objects in most fog, where visible cameras are challenged.
A team of Sandia researchers recently published a paper in Optics Express describing current results from a three-year project to use computational imaging and the science behind how light propagates and scatters in fog to create algorithms that enable sensors to detect, locate and image objects in fog.
“Current methods to see through fog and with scattered light are costly and can be limited,” said Brian Bentz, electrical engineer and project lead. “We are using what we know about how light propagates and scatters in fog to improve sensing and situational awareness capabilities.”
Brian Bentz, electrical engineer and project lead, said the team has modeled how light propagates through fog to an object and to a detector – usually a pixel in a camera – and then inverted that model to estimate where the light came from and characteristics of the object. By changing the model, this approach can be used with either visible or thermal light. Bentz says the team has used the model to detect, locate and characterize objects in fog and will be working on imaging objects during the project’s final year.