一种联合学习局部与全局特征的红外和可见光图像融合方法

Fusion Approach for Joint Learning of Local and Global Features in Infrared and Visible Images

  • 摘要: 现有的红外和可见光图像融合方法不能充分整合局部与全局特征表示,导致融合图像存在明显的偏向性和不平滑性。由此,本文提出了一种联合学习局部与全局特征的融合方法(JLFuse)。首先,在传统卷积采样的基础上引入卷积Transformer,增强对全局特征的建模能力。其次,设计了一种基于空间可分离自注意力的融合策略,利用局部分组自注意力和全局子采样注意力交替引导的Transformer模块,实现了局部与全局特征的联合学习。最后,采用金字塔型的设计原则,获取多尺度特征,并加强局部传播。在TNO和RoadScene数据集上的实验结果表明,与6种先进融合方法相比,所提方法在多个客观评价指标上展现了优越性。在主观上,融合图像更加符合人类视觉偏好。

     

    Abstract: Existing infrared and visible image fusion approaches cannot fully integrate local and global feature representations, resulting in bias and smoothness in the fused image. Therefore, in this study, we propose a fusion approach for jointly learning local and global features, namely JLFuse. First, a convolution transformer is introduced based on traditional convolution sampling to enhance the modeling ability of the global features. Second, a fusion strategy (JLFN), based on spatially separable self-attention, is designed using locally grouped self-attention and global sub-sampled attention alternately guided transformer modules to achieve joint learning of local and global fusion features. Finally, the pyramid design principle is adopted to obtain multiscale features and enhance the local propagation. Experimental results on the TNO and RoadScene datasets show that the proposed approach outperforms six advanced fusion approaches in multiple objective evaluation metrics. Subjectively, the fused images are more consistent with human visual preferences.

     

/

返回文章
返回