NASA engineer Semion Kizhner spent an entire week trying to understand the Hilbert-Huang Transform with the colleague who had developed it, but to no avail. He hoped to develop a software that would get people using this powerful but complex new algorithm, he says, because he saw great potential in it that might require decades of development.
Fortunately, no one needs to fully understand a transform to use it. “I have no clue how intestines work, but that doesn’t prevent me from eating,” says Kizhner, now recently retired from Goddard Space Flight Center’s Instrument Electronics Development Branch.
A transform—essentially, a particular type of mathematical formula—helps scientists gather information from complex data. For example, Kizhner says, he might not be able to interpret the raw data from an electrocardiogram reading, but transforming the data into the frequency domain might show that something is happening about once a second, which is the normal respiration rate—a clue suggesting the phenomenon is probably associated with breathing.
“You try to apply some known transform, and if you’re lucky, it will give you something you understand,” Kizhner says.
One of the most commonly used transforms is the century-old Fourier Transform, originally devised to study heat transfer but now used in applications as diverse as quantum mechanics and digital image compression. The Fourier Transform, however, has severe limitations when it comes to analyzing natural phenomena. Kizhner had been looking for an alternative, and the Hilbert-Huang Transform (HHT), created by Goddard chief scientist for oceanography Norden Huang, seemed like a promising option.
It’s the product of the oceanographer’s nearly four decades of attempts to come up with a transform that could make more sense of data on ocean waves and atmospheric turbulence. It breaks down a one-dimensional signal into a series of what Huang called intrinsic mode functions, isolating various frequencies, and then applies the preexisting one-dimensional Hilbert spectral analysis to decompose frequency components.
Kizhner notes that the Royal Society of London published all of Huang’s 100-page paper on the subject despite the organization’s 10-page limit. “Obviously, someone was impressed.”
Although Kizhner still struggled to understand the mechanics of the transform, he and one of his students, Karin Blank, were able to write software to apply it, with initial funding from Goddard’s Internal Research and Development (IRAD) Program.
The resulting HHT Data Processing System software for one-dimensional data, which can be licensed from Goddard, works well beyond oceanography. “HHT1 works on data like nothing else before it, and people are using it all over the world,” Kizhner says, including to help monitor blood flow in the brain after brain injuries (Spinoff 2010). In 2006, the Federal Laboratory Consortium’s mid-Atlantic region named the software Best Technology of the Year.
But Kizhner—and many others, including Huang himself—were looking for a way to use the transform in two dimensions to analyze images. “There are certain things in image processing that cannot be done well with existing techniques, so we always strive to find something new,” he says.
Such a tool could help detect and eliminate stray light and other extraneous data in, say, long-exposure images of deep space. But it might also be able to extract even deeper, more specific information, Kizhner says. He likens it to using an image of illegible handwritten notes to make determinations about the writer’s character. “It’s very difficult to do.”
Following years of attempts, on a camping trip, Kizhner was finally struck by an idea that would let him apply the Hilbert Transform across two dimensions, paving the way for Huang’s technique to also analyze frequencies across a plane—a near real-time HHT2 algorithm.
It worked. With a little more IRAD funding and some help from coworkers, Kizhner developed and patented a prototype in the MATLAB programming language. To do so, he developed a way to also extend Huang’s empirical mode decomposition to two dimensions—and he sped up the whole thing by writing it as an algorithm that could run on many processing units simultaneously, in near real time. He also implemented a novel Hilbert Transform for two-dimensional spectral analysis.
One of his assignments for his day job was to help design a data processing system for a radiometer to fly on the Soil Moisture Active Passive (SMAP) satellite, which launched in 2015. The radiometer operates in the L1 microwave band, a wavelength that is unaffected by the atmosphere. That same characteristic, however, makes the L1 bandwidth popular for other uses, such as cell phones, which could interfere with the instrument’s signals. Goddard decided to try out the two-dimensional Hilbert-Huang software prototype, using it to remove radio interference from the satellite’s imagery and recover the data that had been contaminated, rather than simply deleting all contaminated data, as state-of-the-art systems did at the time.
Arlington, Virginia-based Syneren Technologies Corporation collaborated on the effort, and after a year of working together closely, the company expressed interest in commercializing the prototype HHT2 program. Syneren currently holds an exclusive field-of-use license.
It was clear the software’s image-processing capability could theoretically be applied to a wide swath of industries, but figuring out the details was better suited to the private sector, Kizhner says.
“I thought it was a great idea for us to continue the research and actually make a commercial product, because if we didn’t, this would just evaporate,” recalls Meg Vootukuru, Syneren’s owner and CEO.
Syneren signed a Space Act Agreement, and Kizhner has consulted with the company weekly ever since.
“They extended the software, enriched it, fixed bugs, and put in a lot of work, and they came up with SIETech,” Kizhner says, or Syneren Image Enhancement Technology.
To begin with, the company rewrote the software in C++, as MATLAB can be difficult to install and use. “We developed an algorithm and software that makes it easy to commercialize,” says Vootukuru, noting that the program can be installed on any hardware platform and leverages multiprocessing to run fast and efficiently.
By the end of 2016, Syneren was in talks with a number of companies and Government agencies about how the software could be tailored to each of their needs. Different clients would need different products, geared toward the solutions they’re looking for, says SIETech product manager Richard Williams. “The product is tailorable—we’ve got to determine what question you’re trying to answer. We’re open to whatever our clients need.”