文章摘要

文章信息/Info

Title:: Fusion algorithm of multi-spectral images based on dual-tree complex wavelet transform and frequency-domain U-Net

Author(s):: LI Jianfei; CHEN Chunxiao; WANG Liang; Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China

Keywords:: Multi-spectral image fusion; Near-infrared fluorescence image; Dual-tree complex wavelet transform; Convolutional neural network; Frequency-domain U-Net

摘要:: 提出基于双树复小波变换（dual-tree complex wavelet transform，DTCWT）和频域U-Net的多光谱图像融合算法。通过将彩色图像和基于荧光示踪剂标记肿瘤等研究目标的荧光图像进行DTCWT分解，获取源图像的高低频分量，利用频域U-Net对荧光图像的低频子带进行处理，分割得到只包含目标区域分布的感兴趣区域。基于感兴趣区域对将荧光图像的高低频子带以低频加权相加、高频取绝对值较大者的融合规则与彩色图像绿色通道融合，并经过DTCWT逆变换重构得到多光谱融合图像。实验表明，本研究提出的多光谱图像融合方法相较于传统融合算法具有更高的空间频率、结构相似性、互信息以及边缘信息保持度，突出了目标区域特征，提高了融合图像的显示效果。

Abstract:: We proposed a novel multi-spectral image fusion algorithm for image fusion based on dual-Tree complex wavelet transform (DTCWT) and U-Net.The color images and near-infrared fluorescence images of tumors or other targets markde probes were processed by DTCWT to acquire high and low frequency information of input images ,then obtained the target region contained regions of interests which were created by frequency-domain U-Net.The components with different frequency of fluorescence images and green channel of color images were combined by different rules. At last, IDTCW was applied to reconstruct the fused image. The experiments show that the algorithm outperforms traditional fusion methods on spatial frequency, structural similarity, mutual information and QABF and highlights the characteristics of targets to improve visual effect ofthe fused image.