image: Comparison of traditional iteration-PSR, physical enhanced neural network (PN-PSR), and the proposed MPPN-PSR method.
Credit: OEA
A new publication from Opto-Electronic Advances; DOI 10.29026/oea.2024.240060, discusses multi-prior intelligent microscopy assisted high-throughput, pixel super-resolution quantitative phase imaging.
The phase information carried by light reveals various characteristics of objects, such as the thickness, refractive index, geometric structures, etc. However, due to the fact that most optical sensors are intensity based devices, phase information cannot be directly detected by such sensors. Digital holographic microscopy (DHM), an interferometric technology, is one of the commonly used methods for non-destructive phase imaging. DHM typically involves two categories: the digital in-line holographic microscopy (DIHM) and the off-axis digital holographic microscopy. The off-axis DHM allows wavefront reconstruction from a single-shot digital hologram, but suffers from the loss of space-bandwidth and the decrease of resolution. In comparison, DIHM has relatively higher space-bandwidth product, and is often preferred in some microscopic scenes. Unfortunately, two factors obfuscate the high-quality reconstruction in DIHM: 1) Interference from twin images during holographic reconstruction process; 2) Subpixel information loss caused by large pixel size detectors for low exposure time but high signal-to-noise ratio conditions.
While both of the aforementioned issues can be addressed to some extent through physical modification in holographic setup or numerical compensation, they often encounter challenges such as increased optical complexity and insufficient robustness under strong noise conditions. In contrast, the deep learning (DL) network, with its noise suppression and inverse problem-solving capabilities, has become a powerful tool for DIHM imaging and Pixel Super-Resolution (PSR). However, most DL-based strategies are data-driven or end-to-end net approaches, suffering from excessive data dependency and limited generalization ability. Although the training-free DL approach combines a complete physical model representing the imaging process with the deep image prior (DIP) has been proposed to solve the phase retrieval issue, it has been found that directly applying the DIP framework with a single prior to solve the ill-posed inverse problems often leads to pseudo-solution or overfitting of interference-related noise and weight decay, especially when down-sampling is encountered simultaneously.
In response to the above challenges, a research team led by Prof. Baoli Yao and Chen Bai at the State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics (XIOPM) of the Chinese Academy of Sciences (CAS), reported a training-free network for quantitative phase retrieval in DIHM imaging with PSR, termed the Multi-Prior Physics-enhanced Neural Network (MPPN-PSR). The network encapsulates the physical model prior, sparsity prior and deep image prior in an untrained deep neural network. Specifically, the physical model priors represent the DIHM imaging process and detector down-sampling, while the sparsity prior further enhances the imaging resolution. Consequently, the retrieval of phase distribution can be achieved with pixel super-resolution, twin-image-free, and insensitivity to noise without any additional hardware design.
As the MPPN-PSR integrates various prior information within a training-free network framework, this approach can effectively and accurately reconstruct phase from a single-shot in-line digital hologram with PSR, avoiding excessive data dependence and limited generalization capabilities associated with traditional end-to-end approaches. MPPN-PSR maximizes the data efficiency by reducing the intensity image redundancy requirement to only one frame, and improves the space-bandwidth product (SBP) since the inherent large FOV of the low-resolution intensity image is exploited. The performance and effectiveness of MPPN-PSR were evaluated by comparing it with other retrieval methods through both simulations and experiments. The pixel resolution of phase imaging can be increased by 3 times compared with the phase retrieval without PSR, while the optical resolution can be improved by about 2 folds compared to the traditional iterative PSR phase retrieval. Given its capability of achieving pixel super-resolution, twin-image elimination, and large-SBP phase reconstruction, the MPPN-PSR method provides high-throughput phase imaging with high accuracy. These superior performances can be widely used in biomedical workflow and industrial measurement.
Keywords: optical microscopy / quantitative phase imaging / digital holographic microscopy / deep learning / super-resolution
# # # # # #
The team led by Prof. Baoli Yao has been engaging in intelligent optical microscopy in recent years, forming notable advancements in expanding the imaging function, the dimensions of information acquisition, the imaging specification, etc.. For example, they use deep learning technology to achieve full-color wide-field microscopic optical sectioning 3D imaging, fast confocal super-resolution 3D imaging, fast light-sheet 3D microscopy, etc.. In addition, the compressed sensing technology is utilized to achieve high resolution and high signal-to-noise ratio light-sheet microscopy, rapid quantitative phase retrieval behind scattering media, etc.. The research results were published in Photon. Res.、Opt. Lett.、Opt. Express and other journals. Besides, the team has conducted long-term theoretical and experimental research on optical microscopy and optical micromanipulation based on light field manipulation. The tem has published more than 300 papers in journals such as PNAS, Nature Com., PRL, Rep. Prog. Phys., Adv. Opt. Photon, and owned numerous invention patents. The team also won the first and second prizes of Shaanxi Provincial Science and Technology Award, as well as awards and honors such as Shaanxi Provincial Key Science and Technology Innovation Team.
# # # # # #
Opto-Electronic Advances (OEA) is a rapidly growing high-impact, open access, peer reviewed monthly SCI journal with an impact factor of 15.3 (Journal Citation Reports for IF2023). Since its launch in March 2018, OEA has been indexed in SCI, EI, DOAJ, Scopus, CA and ICI databases over the time, and expanded its Editorial Board to 34 members from 17 countries.
# # # # # #
More information: http://www.oejournal.org/oea
Editorial Board: http://www.oejournal.org/oea/editorialboard/list
All issues available in the online archive (http://www.oejournal.org/oea/archive).
Submissions to OEA may be made using ScholarOne (https://mc03.manuscriptcentral.com/oea).
ISSN: 2096-4579
CN: 51-1781/TN
Contact Us: oea@ioe.ac.cn
Twitter: @OptoElectronAdv (https://twitter.com/OptoElectronAdv?lang=en)
WeChat: OE_Journal
# # # # # #
Tian X, Li RZ, Peng T et al. Multi-prior physics-enhanced neural network enables pixel super-resolution and twin-image-free phase retrieval from single-shot hologram. Opto-Electron Adv 7, 240060 (2024). doi: 10.29026/oea.2024.240060