Complete depth information can be extracted from analyzing all angles of light rays emanated from a source. However, this angular information is lost in a typical 2D imaging system. In order to record this information, a standard stereo imaging system uses two cameras to obtain information from two view angles. Sometimes, more cameras are used to obtain information from more angles. However, a 4D light field imaging technique can achieve this multiple-camera effect through a single-lens camera.

(a) Two demultiplexed light field images generated by the 4D Light Field Imaging System. The full 4D resolution is 4×4×3039×2014. (b) The estimated depth map of the top image of (a). (c, d) Post-exposure refocused images generated from the light field and the depth maps.
Two methods are available for this: one using a microlens array, and the other using a moving aperture. The moving-aperture method can obtain more complete stereo information. The existing literature suggests a modified liquid crystal panel [LC (liquid crystal) panel, similar to ones commonly used in the display industry] to achieve a moving aperture. However, LC panels cannot withstand harsh environments and are not qualified for spaceflight. In this regard, different hardware is proposed for the moving aperture.

A digital micromirror device (DMD) will replace the liquid crystal. This will be qualified for harsh environments for the 4D light field imaging. This will enable an imager to record near-complete stereo information.

The approach to building a proof-ofconcept is using existing, or slightly modified, off-the-shelf components. An SLR (single-lens reflex) lens system, which typically has a large aperture for fast imaging, will be modified. The lens system will be arranged so that DMD can be integrated. The shape of aperture will be programmed for single-viewpoint imaging, multiple-viewpoint imaging, and coded aperture imaging.

The novelty lies in using a DMD instead of a LC panel to move the apertures for 4D light field imaging. The DMD uses reflecting mirrors, so any light transmission lost (which would be expected from the LC panel) will be minimal. Also, the MEMS-based DMD can withstand higher temperature and pressure fluctuation than a LC panel can. Robotics need near complete stereo images for their autonomous navigation, manipulation, and depth approximation. The imaging system can provide visual feedback.

This work was done by Youngsam Bae of Caltech for NASA’s Jet Propulsion Laboratory. NPO-48604

Topics:
Automation Exteriors Hardware Imaging and visualization Inspections Medical, health and wellness Medical, health, and wellness Mirrors Optics Parts Physical Sciences Powertrains Pressure Robotics Surgical procedures Transmissions
This Brief includes a Technical Support Package (TSP).
Document cover
4D Light Field Imaging System Using Programmable Aperture

(reference NPO-48604) is currently available for download from the TSP library.

Don't have an account?

Magazine cover
NASA Tech Briefs Magazine

This article first appeared in the December, 2012 issue of NASA Tech Briefs Magazine (Vol. 36 No. 12).

Read more articles from this issue here.

Read more articles from the archives here.

Overview

The document discusses the 4D Light Field Imaging System Using Programmable Aperture, developed by NASA's Jet Propulsion Laboratory (JPL) in collaboration with the California Institute of Technology. This innovative imaging system aims to capture complete depth information by analyzing light rays emanating from a source, overcoming the limitations of traditional 2D imaging systems that typically lose angular information.

The system employs two primary techniques for capturing depth information: a micro lens array and a moving aperture. The micro lens array uses an array of pixels beneath each micro lens to analyze the angles of incoming light rays, allowing for a single exposure with a low-resolution imager (1 Mpixel). In contrast, the moving aperture technique can provide more comprehensive stereo information but requires a high-resolution imager (100 Mpixel) and multiple exposures.

A significant innovation in this project is the replacement of traditional liquid crystal panels, which are unsuitable for harsh environments, with a Digital Micromirror Device (DMD) from Texas Instruments. The DMD is a micro-electromechanical system (MEMS) device that can withstand extreme temperatures and pressures, making it suitable for space missions. The proposed hardware aims to generate depth maps with a resolution of 1 mm at a distance of 1 foot from an object, with plans for testing in space-like environments to ensure its qualification for space flight.

The document outlines the approach to integrating the DMD with a modified single-lens reflex (SLR) lens system, which will allow for various imaging modes, including single viewpoint, multiple viewpoint, and coded aperture imaging. This flexibility is expected to simplify the imaging process, potentially replacing larger, more complex payloads with smaller, more efficient units.

The research is conducted under a contract with NASA, emphasizing the potential applications of this technology in various fields, including in-situ instruments and human systems interfaces. The document serves as a technical support package, providing insights into the development and capabilities of the 4D light field imaging system, while also offering contact information for further inquiries related to the technology and its applications.