基于注意力交叉引导的红外与可见光图像融合方法

An Infrared and Visible Light Image Fusion Method Based on Cross-Attention Guidance

  • 摘要: 红外与可见光图像融合旨在将不同模态图像的补充信息集成到单一图像中,提高关注目标的突出特征和纹理细节的利用。然而,现有融合方法难以深入分析和理解这些模态间的复杂关系,导致红外图像在突显目标特征上的优势没有得到充分发挥,可见光图像的纹理细节也未得到有效利用。针对这一问题,提出一种基于注意力交叉引导机制的红外与可见光图像融合方法,其核心思想是通过评估不同模态图像之间的特征相关性,提取关注目标的突出特征和纹理细节特征。模型训练过程中引入了L2范数进行约束,并联合结构重参数化技术来优化模型架构,以提升模型的运行效率和处理能力。在MSRS、TNO和RoadScene三个公开数据集上的实验结果表明,所提模型能够融合不同模态图像之间的深层次特征,有效利用关注目标的突出特征和纹理细节,其视觉信息保真度(VIF)较各数据集上对比方法的最高值分别提高2.04%、3.89%和4.71%。此外,采用结构重参数化后,模型运行时间较重参数化前缩短23.19%。所提方法在提升融合性能的同时提高了运行效率。

     

    Abstract: Infrared and visible image fusion integrates complementary information from different modalities to enhance salient target features and preserve textural details. The existing methods underutilize the strengths of the individual modalities and struggle to capture the complex intermodal relationships. To address this issue, an attention-guided cross-fusion method was investigated to evaluate the intermodal feature correlations and extract salient target features and texture details. L2 norm constraints were introduced during the training, and structural reparameterization optimized the model architecture to improve the efficiency. Experiments on the MSRS, TNO, and RoadScene datasets demonstrated that the model effectively combined deep features across modalities, preserving the salient target information and texture details. The visual information fidelity exceeded the highest values among the compared methods by 2.04%, 3.89%, and 4.71% for the three datasets. Structural reparameterization reduced the runtime by 23.19% relative to the model before reparameterization. The proposed method improved both the fusion performance and computational efficiency.

     

/

返回文章
返回