基于双分支网络的红外与可见光图像融合方法

Infrared and Visible Image Fusion Method Based on a Dual-branch Network

  • 摘要: 在红外和可见光图像融合领域中,传统的图像融合方法仅利用相同的卷积操作来提取局部特征,这可能导致融合结果中语义信息和纹理细节信息的缺失。因此,本文提出了一种双分支网络的图像融合方法。首先,为了提高模型对图像低层次细节和高层次语义的描述能力,将源图像输入到编码器的双分支结构中,通过并行的方式分别提取细节信息和语义信息;其次采用梯度残差密集块强化编码网络,增强模型对细粒度信息的描述能力;特征融合网络采用双边引导聚合层的策略进行两个分支的深度特征融合。最后,在TNO公开数据集上,将所提方法与其他7种融合方法进行了对比和消融实验。结果表明,本方法得到的融合图像结果信息丰富、更契合人体的视觉感知且在峰值信噪比、差异相关性总和、视觉信息保真度等客观指标方面具有明显优势。

     

    Abstract: In the fields of infrared and visible image fusion, traditional image fusion methods only use the same convolution operation to extract local features, which may lead to the absence of semantic and texture detail information in the fused results. Therefore, this paper proposes a two-branch network approach for image fusion. First, to improve the ability of the model in describing low-level details and high-level semantics of the image, the source image is input into the two-branch structure of the encoder, and detail information is extracted separately but in parallel with semantic information. Second, gradient residual dense block-reinforced coding network was used to enhance the model's ability to describe fine-grained information. The feature fusion network uses the strategy of bilateral bootstrap aggregation layer for the deep feature fusion of the two branches. Finally, the proposed method was compared with seven other fusion methods on the TNO public dataset and experimentally ablated. The results show that the fused images obtained by the proposed method are rich in information, more suitable for human visual perception, and have significant advantages in objective indices such as peak signal-to-noise ratio, sum of difference correlation, and visual information fidelity.

     

/

返回文章
返回