A novel deep learning model for medical image segmentation with convolutional neural network and transformer

Accurate segmentation of medical images is essential for clinical decision-making, and deep learning techniques have shown remarkable results in this area. However, existing segmentation models that combine transformer and convolutional neural networks often use skip connections in U-shaped networks, which may limit their ability to capture contextual information in medical images.

To address this limitation, the team led by Hua Bai and Baoshan Sun from Tiangong University proposed a coordinated mobile and residual transformer UNet (MRC-TransUNet) that combines the strengths of transformer and UNet architectures. Their approach uses a lightweight MR-ViT to address the semantic gap and a reciprocal attention module to compensate for the potential loss of details. To better explore long-range contextual information, they use skip connections only in the first layer and add MR-ViT and RPA modules in the subsequent downsampling layers.

They evaluated the effectiveness of our proposed method on three different medical image segmentation datasets, namely, breast, brain, and lung. Their proposed method outperformed state-of-the-art methods in terms of various evaluation metrics, including the Dice coefficient and Hausdorff distance. These results demonstrate that our proposed method can significantly improve the accuracy of medical image segmentation and has the potential for clinical applications.