模态自适应的红外与可见光图像融合

Mode Adaptive Infrared and Visible Image Fusion

  • 摘要: 为解决低照度和烟雾等恶劣环境条件下融合图像目标对比度低、噪声较大的问题,提出一种模态自适应的红外与可见光图像融合方法(mode adaptive fusion, MAFusion)。首先,在生成器中将红外图像与可见光图像输入自适应加权模块,通过双流交互学习二者差异,得到两种模态对图像融合任务的不同贡献比重;然后,根据各模态特征的当前特性自主获得各模态特征的相应权重,进行加权融合得到融合特征;最后,为了提高模型的学习效率,补充融合图像的多尺度特征,在图像融合过程中加入残差块与跳跃残差组合模块,提升网络性能。在TNO和KAIST数据集上进行融合质量测评,结果表明:主观评价上,提出的方法视觉效果良好;客观评价上,信息熵、互信息和基于噪声的评价性能指标均优于对比方法。

     

    Abstract: To solve the problems of low contrast and high noise of fused images in low illumination and smoky environments, a mode-adaptive infrared and visible image fusion method (MAFusion) is proposed. Firstly, the infrared and visible images are input into the adaptive weighting module in the generator, and the difference between them is learned through two streams interactive learning. The different contribution proportion of the two modes to the image fusion task in different environments is obtained. Then, according to the characteristics of each modal feature, the corresponding weights of each modal feature are obtained independently, and the fusion feature is obtained by weighted fusion. Finally, to improve the learning efficiency of the model and supplement the multi-scale features of the fused image, a residual block and jump connection combination module are added to the image fusion process to improve the network performance. The fusion quality was evaluated using the TNO and KAIST datasets. The results show that the visual effect of the proposed method is good in subjective evaluation, and the performance indexes of information entropy, mutual information, and noise-based evaluation are better than those of the comparison method.

     

/

返回文章
返回