The device and method are used to quantify end-to-end latency of head- or helmet-mounted display with head tracking systems in a laboratory or in situ. All commercial or custom head-mounted display systems that track the user’s head for the purpose of virtual or augmented reality applications encounter positional display errors due to system latency. A basic head-mounted display (HMD) with head-tracking system is comprised of (1) a near-to-eye display, (2) the head-tracking system, (3) one or more symbology or image sources, and (4) the display/image processor. Each element, and the communication among them, contributes a portion to the total latency. HMD system latency manifests as erroneous alignment of the virtual and real surroundings as the head is slewed, and is known to induce simulator sickness and other physiological issues. Therefore, minimal system latency is a design goal to reduce these physiological symptoms. The overall latency budget is the sum of time required to measure the dynamic head position, communicate the position to the display processor, compute the scene based on the position, integrate imagery, and render the scene to the display.
The Head-Mounted Display Latency Measurement Rig (HeLMR) device (see figure) consists of a 3D-printed, anatomically approximate human head that is able to “wear” all available commercial and custom HMD systems. A high-speed camera is installed in place of the eye(s) in the correct image plane location. The head is mounted on a precision rotary stage that moves the head in a left-right-left “No-No” fashion at a precise angular rate. The device’s native geo-referenced virtual image is rendered and aligned to overlay a real-world reference. Static and dynamic lasers are used for reference to aid the alignment. The head is moved and tracked dynamically while the high-speed (300 fps) video image from the camera shows the angular skew of the virtual-to-actual image due to the system latency. A snapshot image is captured and the angular difference of the virtual image to the actual image is measured with computer-aided design software. With the error angle measurement and the precision rate information, the total system latency is calculated. The HeLMR device/method quantifies the end-to-end latency of the entire system and can be mounted on a synthetic torso for in situ measurements of flight simulators or other applications.
Initial testing was conducted on a custom HMD for a lunar lander application. As with a standard tracking HMD system, symbology (e.g., terrain, attitude, guidance) is rendered that is referenced to a world coordinate system. As the head is slewed, the symbology is misaligned with the surrounding terrain due to the system latency. This head tracker system uses a hybrid combination of optical and inertial sensors where both the display and tracker are mounted to clear-lens glasses with a head-borne weight of 4 oz. (113 g). The total system latency of the lunar HWD system was measured to be 67 milliseconds with detailed lunar terrain rendering, and 33 milliseconds without terrain. This indicates that the latency is content-dependent in addition to several other factors, including the display update rate. This system is novel with the ability to non-intrusively quantify system latency and has been successfully used on several HMD systems. The apparatus combines precision COTS equipment in a unique arrangement, enabling the measurement of a difficult-to-quantify parameter.
This work was done by Kevin Shelton, Randy Bailey, Trey Arthur, Steve Williams, Lance Prinzel, Lynda Kramer, Denise Jones, and Kyle Ellis of Langley Research Center. LAR-17884-1