融合全局与局部信息的红外-可见光目标检测

Infrared Visible Light Object Detection Integrating Global and Local Information

  • 摘要: 为提升复杂场景下多模态目标检测的性能,提出一种融合全局与局部特征的红外-可见光目标检测模型。模型采用多源异构网络架构进行设计:针对可见光图像,利用基于特征关键度的分组卷积来构建卷积神经网络,通过滑动窗口机制从局部空间维度提取目标细节纹理特征;针对红外图像,采用高效率多头自注意力机制来构建视觉Transformer网络,通过全局上下文建模来捕捉目标显著特征。针对红外-可见光特征交互,设计动态权重自适应融合策略,通过关键特征增强与模态权重分配实现跨模态特征互补。同时,为缓解不同尺度特征融合时的信息冲突,提出空间-通道多尺度协同优化模块,通过注意力机制分别从空间和通道维度突出有效特征并抑制噪声信息,进而降低特征间的相互干扰。通过在多个标准数据集上的实验结果表明:所提方法在多模态特征提取、跨模态特征交互以及多尺度特征融合等方面相较于YOLOv10基线模型均有显著提升,并且与现有同类型网络相比,该方法也体现出更优的检测效果,能够在复杂环境下精准高效地实现目标检测。

     

    Abstract: A model for infrared–visible object detection that integrated global and local features was developed to improve the performance of multimodal object detection in complex scenes. The model was designed using a multisource heterogeneous network architecture. For visible–light images, group convolution based on feature importance was used to build a convolutional neural network, extracting the detailed texture features of objects from local spatial dimensions through a sliding window mechanism. For infrared images, a highly efficient multihead self-attention mechanism was employed to build a visual transformer network, capturing the salient features of objects through global context modeling. To facilitate the interaction of infrared and visible features, a dynamic weight-adaptive fusion strategy was designed to achieve cross-modal feature complementarity through key-feature enhancement and modal-weight allocation. Additionally, a spatial-channel multiscale collaborative optimization module was developed to alleviate information conflicts when fusing features of different scales. The most effective features were highlighted and noise information from the spatial and channel dimensions was suppressed through an attention mechanism, thereby reducing the mutual interference between features. Experiments on multiple standard datasets showed that the proposed method significantly improved multimodal feature extraction, cross-modal feature interaction, and multiscale feature fusion compared to the YOLOv10 baseline model. Moreover, compared to similar existing networks, this method demonstrated better detection performance, enabling accurate and efficient object detection in complex environments.

     

/

返回文章
返回