复杂退化场景下的红外可见光图像融合网络

Infrared And Visible Image Fusion Network In Complex Degraded Scenarios

  • 摘要: 红外与可见光图像融合技术通过整合两种模态的互补信息,在复杂环境下提升图像感知能力,广泛应用于夜视感知、自动驾驶与安防监控等场景。然而,现有融合方法大多假设输入图像质量良好,缺乏对雾霾、雨滴、低照度等实际退化因素的建模能力,导致在真实场景中鲁棒性与适应性较差。构建了一个包含退化感知混合专家模块与模态重建机制的深度融合框架,称为DAME-Fusion。前者依据输入图像的退化类型动态选择去雾、去雨、低光增强等不同专家分支,以增强模态特征;后者通过模态重建分支引导跨模态语义一致性学习,提升融合图像的结构表达与细节保真度。在LLVIP、MSRS、M3FD与RoadScene等公开数据集上,DAME-Fusion在平均梯度(AG)、熵(EN)、空间频率(SF)、标准差(SD)、视觉信息保真度(VIF)和互信息(MI)等多个主流指标上取得最优或次优性能,融合图像在结构还原、细节清晰度和语义一致性方面明显优于现有先进方法。提出的退化感知融合框架能够在多种退化场景下实现鲁棒而高质量的红外与可见光图像融合,验证了多专家机制与模态重建约束在提升多模态感知系统稳定性与泛化能力方面的显著优势,具有良好的实际应用前景。

     

    Abstract: Infrared and visible image fusion aims to integrate complementary information from two modalities to enhance visual perception under adverse conditions, and has been widely applied in night vision, autonomous driving, and surveillance. However, most existing methods assume clean inputs and ignore the presence of complex degradations, such as haze, rain, and low illumination, leading to poor robustness in real-world applications. To address this issue, we propose a novel deep fusion framework named DAME-Fusion, which incorporates a degradation-aware mixture-of-experts (MoE) module and a modality reconstruction mechanism. Specifically, the MoE module adaptively selects enhancement experts based on degradation types (e.g., fog, rain, and low-light), while the reconstruction branch enhances cross-modal consistency. Extensive experiments conducted on four public datasets (LLVIP, MSRS, M3FD, and RoadScene) show that our method achieves state-of-the-art performance across multiple metrics, including average gradient (AG), entropy (EN), spatial frequency (SF), standard deviation (SD), visual information fidelity (VIF), and mutual information (MI). Compared with existing approaches, our method produces fusion results with clearer detail, better structure preservation, and more consistent semantic alignment, validating its effectiveness and practical potential in degraded environments.

     

/

返回文章
返回