Learnable digital signal processing: a new benchmark of linearity compensation for optical fiber communications

The continuous evolution of digital signal processing (DSP) technologies is pivotal for enhancing the capacity of optical fiber communication systems. Traditional DSP frameworks have achieved significant maturity in their block-by-block design. However, the block-by-block nature of DSP design may lead to local performance minima. Additionally, the performance of fiber nonlinear compensation, benchmarked against traditional DSP, could be inaccurate due to interference from linearities. Therefore, attaining optimized performance in linear DSPs is crucial for accurately assessing the advantages offered by nonlinear DSPs, seeking for higher transmission capacity.

 

In a new paper published in Light: Science & Application, a team of scientists, led by Professor Lilin Yi from State Key Lab of Advanced Optical Communication Systems and Networks, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, China, and co-workers have developed an innovative approach known as learnable DSP (LDSP). This new method integrates deep learning optimization within the existing DSP framework, offering substantial improvements in performance and efficiency. They presented their LDSP framework, which reuses traditional DSP modules but treats the entire DSP process as a deep learning structure. By applying backpropagation algorithms, LDSP optimizes DSP parameters globally, resulting in enhanced compensation for both linear and nonlinear performance.

 

In their study, the researchers found that the LDSP framework could achieve several different functions with a single module, showcasing its high efficiency. They also demonstrated that LDSP-enhanced perturbation-based nonlinear compensation exhibits notable nonlinear compensation (NLC) performance due to the increased accuracy in linear compensation, enhancing the precision of nonlinear estimation. Therefore, they highlight the potential of LDSP as a new and highly efficient benchmark for linearity compensation in optical fiber communications. They conducted experimental trials involving 1600 km fiber transmission at 400Gb/s, and the results demonstrated a remarkable improvement in signal performance. Through the proposed optimization, LDSP achieves comprehensive module-level optimization, leading to approximately 0.77 dB and 0.56 dB enhancement in the Q factor for single-channel and 21-channel transmission, respectively. Combined with NLC, there is an observed performance enhancement, achieving a maximum gain of 1.21 dB and 0.9 dB. These scientists summarize the operational principle of their LDSP framework:

 

“While the majority of the DSP modules remain the same, each DSP module is treated as a linear layer of a deep neural network (DNN), and its parameters are optimized using a learning algorithm through backpropagation, specifically the SGD method. This approach enables performance optimization from a global perspective, offering a more holistic and effective solution.”

 

“It is noteworthy that all DSP operations must be implemented differentially and then the backpropagation algorithm can be performed normally. After passing through the entire LDSP module, the loss is computed based on the loss function, and gradients are calculated using the error backpropagation algorithm.” they added.

 

“Due to the DL framework, the LDSP is compatible with DL structures and can be extended to incorporate learnable perspectives for nonlinear compensation in the future. The LDSP could emerge as a new and highly efficient benchmark for linearity compensation, generating significant interest across various domains of nonlinear compensations and beyond.” the scientists forecast.

---

---