Object Detection in Visible Light and Infrared Images Based on Adaptive Attention Mechanism

ZHAO Songpu; YANG Liping; ZHAO Xin; PENG Zhiyuan; LIANG Dongxing; LIANG Hongjun

Volume 46 Issue 4

Apr. 2024

Turn off MathJax

Article Contents

Article Navigation > Infrared Technology > 2024 > 46(4): 443-451

ZHAO Songpu, YANG Liping, ZHAO Xin, PENG Zhiyuan, LIANG Dongxing, LIANG Hongjun. Object Detection in Visible Light and Infrared Images Based on Adaptive Attention Mechanism[J]. Infrared Technology , 2024, 46(4): 443-451.

Citation:

PDF( 2732 KB)

Object Detection in Visible Light and Infrared Images Based on Adaptive Attention Mechanism

Shenzhen Launch Digital Technology Co. Ltd. Chengdu Branch, Chengdu 610000, China

Received Date: 2022-08-30
Rev Recd Date: 2022-09-28
Publish Date: 2024-04-20

Abstract

Abstract

To address the shortcomings of infrared and visible light object detection methods, a detection method based on an adaptive attention mechanism that combines deep learning technology with multi-source object detection is proposed. First, a dual-source feature extraction structure is constructed based on deep separable convolution to extract the features of infrared and visible objects. Second, an adaptive attention mechanism is designed to fully complement the multimodal information of the object, and the infrared and visible features are weighted and fused using a data-driven method to ensure the full fusion of features and reduce noise interference. Finally, for multiscale object detection, the adaptive attention mechanism is combined with multiscale parameters to extract and fuse the global and local features of the object to improve the scale invariance. Experiments show that the proposed method can accurately and efficiently achieve target recognition and localization in complex scenarios compared to similar object detection algorithms. Moreover, in actual substation equipment detection, this method also demonstrates higher generalization and robustness, which can effectively assist robots in completing object detection tasks.
- infrared and visible light,
- object detection,
- deep learning,
- adaptive attention mechanisms

FullText(HTML)

References(24)

References

[1]	王灿, 卜乐平. 基于卷积神经网络的目标检测算法综述[J]. 舰船电子工程, 2021, 41(9): 161-169. https://www.cnki.com.cn/Article/CJFDTOTAL-JCGC202109036.htm WANG Can, BU Leping. Overview of target detection algorithms based on convolutional neural networks[J]. Naval Electronic Engineering, 2021, 41(9): 161-169. https://www.cnki.com.cn/Article/CJFDTOTAL-JCGC202109036.htm
[2]	郝永平, 曹昭睿, 白帆, 等. 基于兴趣区域掩码卷积神经网络的红外-可见光图像融合与目标识别算法研究[J]. 光子学报, 2021, 50(2): 84-98. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB202102010.htm HAO Yongping, CAO Zhaorui, BAI Fan, et al Research on infrared visible image fusion and target recognition algorithm based on region of interest mask convolution neural network[J]. Acta PHOTONICA Sinica, 2021, 50 (2): 84-98 https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB202102010.htm
[3]	刘齐, 王茂军, 高强, 等. 基于红外成像技术的电气设备故障检测[J]. 电测与仪表, 2019, 56(10): 122-126. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201910020.htm LIU Qi, WANG Maojun, GAO Qiang, et al Electrical equipment fault detection based on infrared imaging technology[J]. Electric Measurement and Instrumentation, 2019, 56(10): 122-126. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201910020.htm
[4]	XIA J, LU Y, TAN L, et al. Intelligent fusion of infrared and visible image data based on convolutional sparse representation and improved pulse-coupled neural network[J]. Computers, Materials and Continua, 2021, 67(1): 613-624. doi: 10.32604/cmc.2021.013457
[5]	汪勇, 张英, 廖如超, 等. 基于可见光、热红外及激光雷达传感的无人机图像融合方法[J]. 激光杂志, 2020, 41(2): 141-145. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202002029.htm WANG Yong, ZHANG Ying, LIAO Ruchao, et al. UAV image fusion method based on visible light, thermal infrared and lidar sensing[J]. Laser Journal, 2020, 41(2): 141-145. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ202002029.htm
[6]	ZHANG S, LI X, ZHANG X, et al. Infrared and visible image fusion based on saliency detection and two-scale transform decomposition[J]. Infrared Physics & Technology, 2021, 114(3): 103626.
[7]	王传洋. 基于红外与可见光图像的电力设备识别的研究[D]. 北京: 华北电力大学, 2017. WANG Chuanyang. Research on Power Equipment Recognition Based on Infrared and Visible Images[D]. Beijing: North China Electric Power University, 2017.
[8]	LI H, WU X J. Infrared and visible image fusion using Latent low-rank representation[J]. Arxiv Preprint Arxiv, 2018: 1804.08992.
[9]	HUI L, WU X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
[10]	唐聪, 凌永顺, 杨华, 等. 基于深度学习的红外与可见光决策级融合跟踪[J]. 激光与光电子学进展, 2019, 56(7): 209-216. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201907023.htm TANG Cong, LING Yongshun, YANG Hua, et al. Decision-level fusion tracking of infrared and visible light based on deep learning[J]. Advances in Lasers and Optoelectronics, 2019, 56(7): 209-216. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201907023.htm
[11]	MA J, TANG L, XU M, et al. STDFusionNet: an infrared and visible image fusion network based on salient object detection[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-13.
[12]	杨雪鹤, 刘欢喜, 肖建力. 多模态生物特征提取及相关性评价综述[J]. 中国图象图形学报, 2020, 25(8): 1529-1538. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202008002.htm YANG Xuehe, LIU Huanxi, XIAO Jianli. A review of multimodal biometric feature extraction and correlation evaluation[J]. Chinese Journal of Image and Graphics, 2020, 25(8): 1529-1538. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202008002.htm
[13]	WANG Z, XIN Z, HUANG X, et al. Overview of SAR image feature extraction and object recognition[J]. Springer, 2021, 234(4): 69-75.
[14]	WEI Z. A summary of research and application of deep learning[J]. International Core Journal of Engineering, 2019, 5(9): 167-169.
[15]	Bochkovskiy A, WANG C Y, LIAO H. YOLOv4: Optimal speed and accuracy of object detection[J]. Arxiv Preprint Arxiv, 2020: 2004.10934.
[16]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770-778.
[17]	Howard A, Sandler M, Chen B, et al. Searching for MobileNetV3 [C]//IEEE International Conference on Computer Vision (ICCV), 2020: 1314-1324.
[18]	CHEN H, WANG Y, XU C, et al. AdderNet: Do we really need multiplications in deep learning?[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2020: 1465-1474.
[19]	宋鹏汉, 辛怀声, 刘楠楠. 基于深度学习的海上舰船目标多源特征融合识别[J]. 中国电子科学研究院学报, 2021, 16(2): 127-133. https://www.cnki.com.cn/Article/CJFDTOTAL-KJPL202102004.htm SONG Penghan, XIN Huaisheng, LIU Nannan. Multi-source feature fusion recognition of marine ship targets based on deep learning[J]. Journal of the Chinese Academy of Electronic Sciences, 2021, 16(2): 127-133. https://www.cnki.com.cn/Article/CJFDTOTAL-KJPL202102004.htm
[20]	Hassan E. Multiple object tracking using feature fusion in hierarchical LSTMs[J]. The Journal of Engineering, 2020(10): 893-899.
[21]	LIN T Y, Dollar P, Girshick R, et al. Feature pyramid networks for object detection[C]// IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 936-944.
[22]	LIU S, HUANG D, WANG Y. Learning spatial fusion for single-shot object detection[J]. Arxiv Preprint Arxiv, 2019: 1911.09516v1.
[23]	LI C, ZHAO N, LU Y, et al. Weighted sparse representation regularized graph learning for RGB-T object tracking[C]// Acm on Multimedia Conference, ACM, 2017: 1856-1864.
[24]	XIAO X, WANG B, MIAO L, et al. Infrared and visible image object detection via focused feature enhancement and cascaded semantic extension[J]. Remote Sensing, 2021, 13(13): 2538. doi: 10.3390/rs13132538