Ultracompact Meta-imagers for Arbitrary All-optical Convolution

Artificial intelligence (AI) has recently gained rapid development in academy and industry due to intense research of deep convolutional neural network (CNN) with a multilayer architecture. In each layer, numerous convolutional operators with functionality-assigned kernels are implemented to extract important features of objects for identification, but they are extremely time-consuming with the increment of AI tasks. In comparison, photons as massless bosons allow lossless propagation and manipulation of light through large-bandgap transparent materials for optical parallel analogue computing without analogue-to-digital and digital-to-analogue convertors, hereby enabling high-speed and low-consumption computation.

Currently, all-optical convolutional computing has two main approaches: Fourier spatial filtering and Green’s function (GF). Fourier spatial filtering configuration with multiple elements is not preferred in integrated photonic systems, meanwhile the complex filter with both amplitude and phase modulation is mandatorily needed for an arbitrary convolutional operation but unachievable for most traditional optical elements. The GF approaches implement optical analog computing by modulating angle-dependent transmittance (or reflection). Although some GF approaches operating in a single device enable high integration, they have the limited angular responses, which are insufficient for arbitrary convolutional computing in AI and image processing.

Since both all-optical approaches have the drawbacks of either low integration or limited functionality, wavelength-multiplexing technologies combining optical and electronic operations have recently been demonstrated to accelerate convolutional computing in neural networks, exhibiting a significant enhancement in speed comparing with electronic computing. However, the electronic parts in such optoelectronic systems still constrain the ultimate speed.

In a new paper published in Light Science & Application, Huang and his co-workers have developed a meta-imager to perform arbitrary all-optical convolution. This meta-imager contains two parts: a metalens for image formation and the other complex-amplitude meta-modulator for reshaping its point spread function (PSF). Wave theory of light predicts image formation in a lens system as a convolutional operation between optical field of an object and the PSF of the lens. With the PSF working as a kernel, the imaging system offers a natural choice for convolutional operation in a parallel, analog and low-consumption way. However, the PSF in the imaging system usually has the fixed pattern such as Airy spot, which cannot support arbitrary operation required in imaging processing and CNN. To realize an arbitrary convolutional operation, they propose a meta-imager, composed of a metalens and a complex-amplitude meta-modulator. The point spread function PSF_meta of meta-imager is the convolution between the point spread function PSF of the system without modulator and the convolution operator H, so the result of an object U₀ through the meta-imager system is U₀ÄPSF_meta= U₀ÄPSFÄH= U¢ÄH, where U¢ is the imaging result only through the metalens, which can realize arbitrary convolution operation of the object U₀.

“We have successfully demonstrated multiple convolutional operations, such as spatial differentiation, denoising, edge detection and enhancement, to improve imaging quality of optical (phase- and amplitude-type) and biological samples (e.g., chromosome molecules, onion and oral epidetmal cells) with different magnifications and spatial resolutions. Such a combination of both multiple functionalities and compact volume is not possible in other approaches.” the scientist summarized.

“In order to reflect the high integration of meta-imager, this paper combines metalens and complex amplitude modulator into a single complex amplitude component, which has the functions of imaging and modulator at the same time. However, the cost of singlet meta-imager is that the output field of view is reduced, and the field of view increases with the increase of d. An effective method to evaluate the field of view of meta-imager system is proposed by correlating the field of view with parabolic phase.” they added.

“All the demonstrations for edge detection are achieved in parallel at the speed of light, leaving the processing time of ~10^-11 seconds, enhanced by 9 orders of magnitude compared with electronic digital convolutions. It therefore enables real-time edge detection of an object in motion. The meta-imager can be extended to artificial intelligence and high-performance computing,” the scientists forecast.  

---

---