When flying the increasingly crowded skies, pilots need to have an arsenal of information: altitude, airspeed, fuel level, distance to their destination, and the location of other planes in the sky. All of this information is presented in a series of two-dimensional instruments, panels, and readouts, meaning the pilot has to mentally assemble the information and translate that into the 3D world to better understand the relationship among air, ground, and traffic. NASA has long been interested in making it as easy as possible for pilots and astronauts to have the best information available to ensure safe flights, knowing that humans are imperfect creatures.
Kyle Ellis is an aerospace research engineer in the Crew Systems and Aviation Operations Branch at Langley Research Center, and spends a good deal of time analyzing pilot feedback to determine which tools and alarms best capture pilots’ attention during flight. “We’ll look at their eye tracking and their brain activity to see what the state of the pilot currently is,” he explained. “Let’s say they haven’t looked at the airspeed indicator in the last five minutes, and the auto-throttle has been disconnected and the plane starts bleeding off speed. The plane is always monitoring what’s going on, but it also makes the assumption that the pilot knows what’s going on as well.”
Alarms and flashing lights are used to draw the pilots’ attention when something’s amiss, but if it were possible to monitor what kind of alert worked best, new tools could be developed to further boost pilots’ awareness of their conditions and better ensure safe travel. “We can do a traditional blink, like we do for a warning, or we can do a 3D blink; have it pop out of the screen at them,” Ellis said. Fighter pilots have head-up displays that require the head to be within a certain space, also known as a head box. This is similar to what is traditionally required for 3D monitors, and is easier for fighter pilots to maintain for the duration of their short flights because they’re strapped into place and don’t move around much. Commercial aviation pilots move quite a bit more, and would often fall outside a head box, making 3D displays non-viable in commercial operations.
NASA has been working with Rochester, NY-based Dimension Technologies Inc. (DTI) to develop a 3D monitor that would not only provide a vivid image, but could track the pilot’s eye movements, all without requiring glasses. The ability to track and follow movement without losing resolution is DTI’s latest advancement, and could be key to helping pilots. In the earlier iterations of the monitor, there were limits to the technology that prevented a pilot from having the kind of full immersion needed when attempting to use 3D displays. Earlier iterations of the technology couldn’t be considered for installation in airplanes because the images did not have the ability to move with the viewer.
Through Small Business Innovation Research (SBIR) contracts, DTI has been working on monitors that can switch between 2D and 3D imaging. By modifying the backlighting system in any LCD monitor and incorporating a forward-facing camera targeting the eyes of the person looking at the monitor, the display follows the movement and position of the viewer in real time without compromising resolution or depth of field. The 3D capability without glasses, thanks to eye-tracking technology, should help NASA ensure pilots have all the information they need to navigate increasingly busy airports.
Games built to be played or viewed in 3D rely on glasses, or players must remain in a “sweet spot” to get the full effect, but DTI has eliminated both those constraints. Players can walk around, and the monitor will track them and keep the 3D at their eyes. There is no loss of resolution or brightness, no restriction in head movement, and it works for two players.
DTI’s latest display, sold commercially as the DTI Mission Critical 2D/3D monitor, has also been met with enthusiasm from 3D animators. Finally, there has been interest from car manufacturers as well, which could lead to 2D/3D displays as part of the center console, driver instrument cluster, and backseat entertainment packages.