双分支特征驱动的可见光与红外光图像融合算法

DFDFuse: Dual-branch Feature-driven Visible and Infrared Image Fusion Algorithm

  • 摘要: 基于深度学习的模型虽能有效处理多模态特征融合,但在低照度环境下通过红外光补偿可见光时易造成细节丢失,限制了图像质量的进一步提升。为此本文提出CNN-Transformer双分支特征驱动的融合网络(DFDFuse),在编码部分采用双分支渐进式特征处理方式,具体来说,利用可逆神经网络与卷积结合处理局部特征,使用Avgformer处理全局低频信息,加强模型对目标细节的关注和提高跨模态细节信息的提取,其中注意力计算被平均池化操作代替;为了提高融合后显著特征表达并抑制噪声干扰,在融合层中加入了差分特征提取(DFE)和融合特征增强( FFE)模块用来增强差异化细节信息。多数据集实验表明,本文方法能有效还原可见光纹理细节和保持热辐射目标信息,实验结果都全面超过CDDFuse,并在定性评价上的7个指标提升10%左右。

     

    Abstract: Although deep learning-based models can effectively handle multi-modal feature fusion, they often suffer from detail loss when compensating visible light with infrared in low-light environments, which limits further improvement of image quality. To address this issue, this paper proposes a CNN-Transformer dualbranch feature-driven fusion network (DFDFuse). The encoding stage employs a dual-branch progressive feature processing mechanism: specifically, a combination of reversible neural networks and convolution operations handles local features, while the Avgformer (with attention computation replaced by average pooling) processes global low-frequency information. This design enhances the model's focus on target details and improves cross-modal detail extraction. To strengthen the expression of salient features and suppress noise interference, we introduce a Differential Feature Extraction (DFE) module and a Fusion Features Enhancement (FFE) in the fusion layer to amplify differentiated detail information. Extensive experiments on multiple datasets demonstrate that our method effectively preserves visible-light texture details while maintaining thermal radiation target information. The proposed approach comprehensively outperforms CDDFuse, achieving approximately 10% improvement across seven qualitative evaluation metrics.

     

/

返回文章
返回