image: Construction of the meta-imaging camera and board depth estimation experiments. a, The meta-imaging camera integrates MLA, CMOS sensor, and piezo stage together. b, Different views of the meta-imaging camera’s 4D PSF. c, A 2D camera and a meta-imaging camera, both possessing identical optical parameters. d, A board with “H”. e, Experimental board images at different distances of the meta-imaging camera. Set the meta-imaging camera focus at 2.48m. f. Estimated depth of the board using deconvolution and PSF model. The “geo error” refers to theoretical depth estimation error based on geometry optics.
Credit: by Cao, Z., Li, N., Zhu, L. et al.
Depth sensing plays a crucial role in various applications, including robotics, augmented reality, and autonomous driving. Monocular passive depth sensing techniques have come into their own for their cost-effectiveness and compact design, offering an alternative to the expensive and bulky active depth sensors and stereo vision systems. While the light-field camera can address the defocus ambiguity inherent in 2D cameras and achieve unambiguous depth perception, it compromises the spatial resolution and usually struggles with the effect of optical aberration. Hence, they are difficult to achieve accurate and robust monocular depth sensing.
In a new paper published in Light: Science & Applications, a team of scientists, led by Professor Hui Qiao from Institute for Brain and Cognitive Sciences and Department of Automation, Tsinghua University, Beijing 100084, China, and co-workers have presented a compact meta-imaging camera and an analytical framework for the quantification of monocular depth sensing precision by calculating the Cramér–Rao lower bound of depth estimation. Quantitative evaluations reveal that the meta-imaging camera exhibits not only higher precision over a broader depth range than the light-field camera but also superior robustness against changes in signal-background ratio. Moreover, both the simulation and experimental results demonstrate that the meta-imaging camera maintains the capability of providing precise depth information even in the presence of aberrations. Showing promising compatibility with other point-spread-function engineering methods, they anticipate that the meta-imaging camera may facilitate the advancement of monocular passive depth sensing in various applications.
The meta-imaging camera integrates the main lens, microlens array, CMOS sensor, and piezo stage. By incorporating a scanning mechanism, the meta-imaging camera can overcome the trade-off between spatial and angular resolution and achieve multisite aberration correction through digital adaptive optics techniques (DAO). Hence, it can optically capture depth information even in the presence of aberrations, ensuring accurate and robust depth sensing. These scientists summarize the operational principle of their camera:
“we present a compact meta-imaging camera and an analytical framework for the quantification of monocular depth sensing precision. Our results reveal that the meta-imaging camera outperforms the traditional light-field camera, exhibiting superior depth sensing capabilities and enhanced robustness against changes in signal-background ratio. Simulation and experimental depth estimation results further confirm the robustness and high precision of meta-imaging cameras in challenging conditions caused by optical aberrations. ”
“The meta-imaging camera complements rather than contradicts stereo vision. It can enhance the depth sensing performance when replacing 2D cameras with meta-imaging cameras in current stereo vision systems” they added.
“This technique could significantly expand the utility of passive depth sensing in challenging scenarios such as autonomous driving, unmanned drones, and robotics, where accurate and robust depth sensing is crucial. Additionally, this breakthrough opens new avenues for future advancements in long-range passive depth sensing, overcoming the limitations previously imposed by optical aberrations. ” the scientists forecast.
Article Title
Aberration-robust monocular passive depth sensing using a meta-imaging camera