4DLatLRR结合双路径生成对抗网络的红外与可见光图像融合

4DLatLRR Combined with Dual-Path Generative Adversarial Network for Infrared and Visible Image Fusion

  • 摘要: 利用深度学习技术实现红外与可见光图像融合已经涌现出许多优秀研究成果,但在保留源图像信息方面容易出现特征信息提取不平衡、不充分等问题,导致信息失真或视觉上不自然。针对以上问题,本文提出了一种四层深度潜在低秩表示结合双路径生成对抗网络的图像融合方法。首先将源图像分解为低秩图像和显著图像,低秩层利用具有多重分类的双路径生成对抗网络同时估计红外低秩和可见光低秩区域的分布;显著层通过设计基于核范数的融合策略,灵活保留和突出局部纹理等特征,同时引入梯度图像增加细节信息。此外,设计主、辅损失来约束生成器提取梯度和强度信息:提取强度信息时,以红外图像为主,补充可见光图像存在的少量强度信息;提取梯度信息时,以可见光图像为主,补充红外图像存在的少量梯度信息。实验结果表明:该方法在5种重要指标均明显优于其他方法,且差异相关性总和指标也处于中上游水平。

     

    Abstract: The use of deep learning techniques for infrared and visible image fusion has produced numerous outstanding research results. However, challenges remain in preserving the source image information, as issues such as imbalances and insufficient feature extraction can lead to information distortion or visually unnatural results. In this study, a method combining a four-layer deep latent low-rank representation with a dual-path generative adversarial network for image fusion is proposed. First, the source image is decomposed into low-rank and salient images. The low-rank layer uses a dual-path generative adversarial network with multiple classifications to estimate the distributions of the infrared and visible low-rank regions simultaneously. The salient layer employs a fusion strategy based on the nuclear norm to flexibly retain and highlight local textures, while introducing gradient images to enhance the detailed information. In addition, primary and auxiliary loss functions are designed to constrain the generator during the extraction of gradient and intensity information. During intensity extraction, the infrared image is primarily used to complement the limited intensity information in the visible image, whereas during gradient extraction, the visible image is primarily used to supplement the limited gradient information in the infrared image. The experimental results demonstrate that this method significantly outperforms the others across five key metrics, with the overall difference correlation sum indicator ranking at the upper-middle level.

     

/

返回文章
返回