NASA Technology

“People told me, ‘You’re an idiot to work on this,’” Eric Fossum recalls of his early experiments with what was at the time an alternate form of digital image sensor at NASA’s Jet Propulsion Laboratory (JPL).

Phone on camera mode
Cell phone cameras, which require tiny, highly efficient image sensors, ended up being the main driver for the mass production of CMOS active pixel sensors. Whenever you take a picture with your phone—or, nowadays, virtually any other digital imager—you’re using NASA technology.

His invention of the complementary metal oxide semiconductor (CMOS) image sensor would go on to become the Space Agency’s single most ubiquitous spinoff technology, dominating the digital imaging industries and enabling cell phone cameras, high-definition video, and social media as we know it.

Imaging devices based on metal oxide semiconductor devices had been attempted since the 1960s, but no one had ever succeeded in making the technology marketable. The little signal amplifiers had long been used in computer circuitry, but imagers using CMOS as sensors suffered from signal noise, among other problems.

Instead, a different imaging technology, using sensors based on the charge coupled device (CCD), allowed high-quality digital photography to come of age by the late 1980s. These image sensors comprise an array of photodetecting pixels that collect charges when exposed to light and transfer those charges, pixel to pixel, to the corner of the array, where they are amplified and measured.

While CCD sensors are capable of producing scientific-grade images, though, they require a lot of power and extremely high charge-transfer efficiency. These difficulties are compounded when the number of pixels is increased for higher resolution or when video frame rates are sped up.

Fossum was an expert in CCD technology—it was why JPL hired him in 1990—but he believed he could make digital images with smaller and lighter machinery using CMOS technology to create what he called active pixel sensors (APS) (Spinoff 1999, 2002, 2010).

GoPro camera on mount, helmet and hand held
The action camera company GoPro takes maximum advantage of the small size and high efficiency of CMOS digital image sensors, using the technology to build tiny, high-definition video cameras that users can affix to themselves, selfie sticks, or surfboards to capture their adventures in high fidelity.

CMOS technology in general had improved since earlier attempts at using it for image-sensing, and Fossum hit on an approach to reduce the signal noise that had plagued earlier imagers, applying a technique called intra-pixel charge transfer with correlated double sampling—something already used in CCDs. Using this technique, he measured a pixel’s voltage both before and after an exposure. “It’s like when you go to the deli counter, and they weigh the container, then weigh it again with the food,” he explains. The sampling corrected for the slight thermal charges and transistor fluctuations that are latent in photodetector readout, and it resulted in a clearer image.

Because CMOS pixels are signal amplifiers themselves, they can each read out their own signals, rather than transferring all the charges to a single amplifier. This lowered voltage requirements and eliminated charge transfer-efficiency issues. And it had the added benefit of allowing almost all the other camera electronics to be integrated onto the computer chip with the pixel array using conventional CMOS production processes, a development that would make CMOS-APS imagers more compact, reliable, and inexpensive.

The very idea of digital photography was dreamed up at JPL by engineer Eugene Lally in the 1960s. Now the concept of a digital camera on a chip shared the same birthplace.

Technology Transfer

By 1993 Fossum and his team knew they were onto something that could be huge for NASA missions and consumer electronics alike, but as they took their findings on the road, giving talks and publishing papers, they met with resistance from the digital imaging industry and even colleagues at JPL. Fossum attributes this skepticism both to earlier failures in CMOS imaging and to people’s instinct to protect their own livelihoods.

“Even a lot of my friends were negative,” he says. “The technology was basically trying to eat their lunch.”

Mother and daughter surfing
GoPro cameras, which leverage the small size and high efficiency of NASA-invented CMOS active pixel sensors, were originally conceived as surfboard-mounted video cameras, an application that remains popular today.
Image courtesy of Kat Patterson

Despite early doubts about CMOS’s potential, several companies signed Technology Cooperation Agreements with JPL and partnered with Fossum and his colleagues to develop the technology.

In 1995, Fossum became the first JPL scientist to license his own invention from the California Institute of Technology (Caltech), which manages the lab, as he, his then-wife and JPL colleague Sabrina Kemeny, and two other JPL coworkers founded a company, Photobit, to develop custom sensors. Caltech’s Technology Transfer Office was created that year, and the office granted Photobit an exclusive license.

“It was sort of the breakthrough spinoff that showed we could do tech transfer out of JPL, too, not just Caltech,” says Fred Farina, the university’s chief innovation and corporate partnerships officer. “So it was the pioneer, in terms of spinoffs out of JPL.”

The following year, Fossum left JPL to become the company’s full-time technological lead. In addition to designing custom sensors, Photobit licensed technology to companies like Kodak and Intel, although most of those early licenses didn’t lead to product lines. By 1997, however, CMOS was being taken seriously, and several companies invested in Photobit, including Schick Technologies, which also obtained—and still holds—an exclusive license for CMOS for dental imaging.

That same year, Sandor Barna, now vice president of core technologies at GoPro, finished his graduate degree and took a job as an engineer at Photobit.

“It was a great example of a truly disruptive technology,” he says, noting that CMOS did not yet perform as well as CCD imagers, but the potential to improve was clear. In addition, it promised to be easier to use with far lower power and could be more cheaply manufactured, he says.

While Photobit held an exclusive license to the technology developed at JPL and filed more than 100 of its own patents, company leadership was concerned that defending its intellectual property would prove difficult as several electronics giants began developing their own CMOS imagers.

Anticipating heavy competition, in 2001 the founders sold Photobit to Micron Technology, which could bring more resources and manufacturing capability to bear. By then, the company—and subsidiary Photobit Technology Corporation, created to handle custom-design contracts—had built a healthy business for itself, and CMOS’s takeover of the imaging industry had begun.