Is that a Smartphone or Microscope in Your Pocket? It's Both!

Andrew Corselli

Researchers have developed a new smartphone-based digital holographic microscope that enables precision 3D measurements. The highly portable and inexpensive microscope could help bring 3D measurement capabilities to a broader range of applications, including educational uses and point-of-care diagnostics in resource-limited settings.

Holographic microscopes digitally reconstruct holograms to extract detailed 3D information about a sample, enabling precise measurements of the sample’s surface and internal structures. However, existing digital holographic microscopes typically require complex optical systems and a personal computer for calculations, making them difficult to transport or use outdoors.

“Our digital holographic microscope uses a simple optical system created with a 3D printer and a calculation system based on a smartphone,” said Research Team Leader Yuki Nagahama from the Tokyo University of Agriculture and Technology. “This makes it inexpensive, portable and useful for a variety of applications and settings.”

In the journal Applied Optics, the researchers demonstrate the smartphone-based digital holographic microscope’s ability to capture, reconstruct, and display holograms in almost real time. The user can even use a pinch gesture on the smartphone screen to zoom in on the reconstructed hologram image.

“Since our holographic microscope system can be built inexpensively, it could potentially be useful for medical applications, such as diagnosing sickle cell disease in developing countries,” said Nagahama. “It could also be used for research in various field environments or in education by allowing students to observe living organisms at school and at home.”

Fast Smartphone-Based Reconstruction

Digital holographic microscopes work by capturing the interference pattern between a reference beam and light scattered from the sample. The hologram is then digitally reconstructed, which generates 3D information that can be used to measure the sample’s features, even those below the surface.

Although smartphone-based digital holography microscopes have been developed previously, available technologies either reconstruct the holograms on a separate device or lack real-time reconstruction. This limitation arises from the restricted computing and memory capacity of most smartphones. To achieve fast reconstruction on a smartphone, the researchers used an approach called band-limited double-step Fresnel diffraction to calculate the diffraction patterns. This method reduces the number of data points, enabling faster computational image reconstruction from holograms.

“When I was a student, I worked on portable digital holographic microscopes, which initially used laptops as the computing system,” said Nagahama. “With the rise of smartphones, I began exploring their potential as computing systems for broader applications and considered leveraging them for tasks like removing artifacts from observed images, which ultimately shaped the development of this microscope.”

To help with portability, the researchers created a lightweight housing for the optical system using a 3D printer. They also developed an Android-based application to reconstruct the holograms acquired by the optical system.

The microscope generates a reconstructed image of the hologram on the image sensor of a USB camera built into the optical system. This hologram can be observed by the Android smartphone, which provides computational image reconstruction in real time. The reconstructed hologram is then displayed on the smartphone, where users can interact with it via the touchscreen.

Here is an exclusive Tech Briefs interview, edited for length and clarity, with Nagahama.

Tech Briefs: What was the biggest technical challenge you faced while developing this smartphone-based microscope?

Nagahama: The biggest technical challenge was the issue of calculation speed. Compared to personal computers, smartphones have limited computing resources and memory capacity. For this reason, the convolutional diffraction calculation algorithm that has been used in personal computer-based calculation systems up to now has not been able to achieve the expected calculation speed. Therefore, we have improved the calculation speed by adopting “Band-limited double-step Fresnel diffraction,” which is a diffraction calculation algorithm that uses fewer calculation resources.

However, the current calculation speed is still unsatisfactory. Therefore, we are examining whether it is possible to perform calculations at higher speeds using the GPUs built into smartphones.

Tech Briefs: How did this project come about? What was the catalyst for your work?

Nagahama: The idea for a portable digital holographic microscope was based on a research topic conducted in my laboratory when I was a student. At that time, laptop PCs, not smartphones, were used as the computing system. Recently, as the performance of smartphones has improved, I came up with the idea that a more sophisticated system could be created by using smartphones as the computation system, and this was the impetus for this project.

Tech Briefs: Can you explain in simple terms how it works?

Nagahama: This digital holographic microscope system employs Gabor-type optics [see image below]. In Gabor optics, two types of light, one transmitted through an object (object light) and the other transmitted through an area without an object (reference light), enter the image sensor and are recorded as interference fringes. These interference fringes are called holograms, and information about the object light is recorded in the holograms.

The object is then observed by performing light diffraction calculations on the hologram to simulate the state of the object light at the position where the object is located.

Tech Briefs: Do you have plans for any further research/work/etc. on the horizon?

Nagahama: To improve the image quality of the observed image in this smartphone-based microscope, we are first considering the removal of the unintended second image. This unintended second image, called the conjugate image, is caused by the hologram recorded by the image sensor having only light intensity information. In the method we are currently studying, we generate holograms for which both the intensity and phase information of the light is known through simulations based on diffraction calculations of the light.

Then, using these holograms as training data for deep learning, we train the deep learning model to transform holograms with only light intensity information into holograms with both light intensity and phase information. The learned model is used to complement the phase information of the holograms recorded by the image sensor to remove the conjugate image. The results of this method have been published in a paper  , and we are currently investigating how to implement this method in smartphones.

Tech Briefs: Do you have any advice for engineers/researchers aiming to bring their ideas to fruition, broadly speaking?

Nagahama: It would be good to have a prototype, even if it is a simple one, on the idea you would like to realize. It is easier to get people interested in your idea and to give you advice if you have a prototype.

---