Low-Visibility Road Target Detection Algorithm Based on Infrared and Visible Light Fusion
-
摘要: 红外图像与可见光图像均被广泛用于目标检测领域,但单模态图像难以满足低能见度道路目标检测的需求,因此本文从双模态融合角度出发,提出一种基于红外可见光融合的低能见度道路目标检测算法。首先,运用多种红外可见光双模态图像融合算法对输入图像进行预处理,并对融合图像的均值、标准差、信息熵、平均梯度、空间频率等5个参数进行定量分析,然后,优化训练检测网络得到针对低能见度道路目标的检测模型,最后,从模型训练结果和目标检测结果的角度评估算法和模型的正确性。实验结果表明,本文算法训练出的模型误检率和漏检率较其他算法明显降低,检测精度较现有算法使用单模态图像由75.51%提升到88.86%,且图像处理速度能够满足实时检测的需求。Abstract: Both infrared and visible images are widely used in the field of target detection; however, unimodal images find it difficult to satisfy the requirements of low-visibility road target detection. Therefore, this study proposes a low-visibility road target detection algorithm based on infrared-visible fusion from the perspective of bimodal fusion. First, the input images were pre-processed using various IR-visible dual-mode image fusion algorithms, five parameters, including mean, standard deviation, information entropy, mean gradient, and spatial frequency of the fused images, were quantitatively analyzed, and the detection model for low-visibility road targets was obtained by optimizing the training detection network. Finally, the accuracies of the algorithm and model were evaluated in terms of the model-training results and target detection results. The experimental results demonstrate that the false- and missed-detection rates of the model trained by the algorithm in this study were significantly reduced compared with other algorithms, and the detection accuracy was improved from 75.51% to 88.86% compared with the existing algorithm using unimodal images; in addition, the image processing speed satisfied the requirement for real-time detection.
-
Key words:
- infrared image /
- low visibility /
- fusion algorithm /
- road target detection
-
表 1 双模态图像融合后图像信息对比
Table 1. Comparison of image information after bimodal image fusion
E SD EN AG SF Tif 46.416 25.285 5.473 1.987 6.853 Wavelet 45.818 23.694 5.401 2.090 6.115 This Article 64.176 28.444 5.326 2.355 7.185 表 2 IoU=50时各项指标对比
Table 2. Comparison of various indicators when IoU=50
p r F1 mAP Visible Light 0.62 0.56 0.57 0.55 Tif 0.75 0.71 0.64 0.68 Wavelet 0.76 0.78 0.75 0.79 This Article 0.81 0.78 0.78 0.81 -
[1] WANG Z G, ZHAN J, LI Y, et al. A new scheme of vehicle detection for severe weather based on multi-sensor fusion[J]. Measurement, 2022, 191: 14. [2] MIAO X, LI S, SHEN H, et al. Moving vehicle detection and tracking using monocular vision[J]. International Journal of Vehicle Safety, 2014, 7(3/4): 425-439. doi: 10.1504/IJVS.2014.063247 [3] 李章维, 胡安顺, 王晓飞. 基于视觉的目标检测方法综述[J]. 计算机工程与应用, 2020, 56(8): 1-9. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202008002.htmLI Z, HU A, WANG X. Survey of vision based object detection methods [J]. Computer Engineering and Application, 2020, 56(8): 1-9. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202008002.htm [4] 崔晓荣, 沈涛, 黄建鲁, 等. 基于BEMD改进的视觉显著性红外和可见光图像融合[J]. 红外技术, 2020, 42(11): 1061-1071. http://hwjs.nvir.cn/article/id/c89c0447-6d07-4a75-99f6-1bf8681cf588CUI X R, SHEN T, HUANG J L, et al. Infrared and visible image fusion based on BEMD and improved visual saliency[J]. Infrared Technology, 2020, 42(11): 1061-1071. http://hwjs.nvir.cn/article/id/c89c0447-6d07-4a75-99f6-1bf8681cf588 [5] 王文卿, 马笑, 刘涵. 基于联合低秩稀疏分解的红外与可见光图像融合[J]. 信号处理, 2021, 37(9): 1770-1780. https://www.cnki.com.cn/Article/CJFDTOTAL-XXCN202109021.htmWANG W Q, MA X, LIU H. Infrared and visible image fusion via joint low-rank and sparse decomposition[J]. Journal of Signal Processing, 2021, 37(9): 1770-1780. https://www.cnki.com.cn/Article/CJFDTOTAL-XXCN202109021.htm [6] 武圆圆, 王志社, 王君尧, 等. 红外与可见光图像注意力生成对抗融合方法研究[J]. 红外技术, 2022, 44(2): 170-178. http://hwjs.nvir.cn/article/id/7f2ae6e4-af9c-4929-a689-cb053b4dda85WU Y Y, WANG Z S, WANG J Y, et al. Infrared and visible image fusion using attention- based generative adversarial networks[J]. Infrared Technology, 2022, 44(2): 170-178. http://hwjs.nvir.cn/article/id/7f2ae6e4-af9c-4929-a689-cb053b4dda85 [7] REN S, HE K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39: 1137-1149. [8] HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans Pattern Anal Mach Intell. , 2015, 37(9): 1904-1916. [9] WEI L, Dragomir A, Dumitru E, et al. SSD: single shot multibox detector[J/OL]. ECCV, 2016: 21-37, Doi: 10.1007/978-3-319-46448-0_2. [10] Bochkovskiy A, WANG C Y, LIAO H. YOLOv4: optimal speed and accuracy of object detection[J/OL]. Computer Science, 2020, arXiv: 2004.10934. [11] ZHAO Q, SHENG T, WANG Y, et al. M2Det: A single-shot object detector based on multi-level feature pyramid network[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 9259-9266. [12] 许德刚, 王露, 李凡. 深度学习的典型目标检测算法研究综述[J]. 计算机工程与应用, 2021, 57(8): 10-25. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202108003.htmXU D G, WANG L, LI F. Review of typical object detection algorithms for deep learning [J]. Computer Engineering and Application, 2021, 57(8): 10-25. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202108003.htm [13] 任全会, 孙逸洁, 黄灿胜. 基于区域间相似度的红外与可见光图像融合算法研究[J]. 红外技术, 2022, 44(5): 492-496. http://hwjs.nvir.cn/article/id/0101943c-987d-48e8-ba4d-1748f31d4bbfREN H Q, SUN Y J, HUANG C S. Infrared and visible image fusion algorithm based on regional similarity [J]. Infrared Technology, 2022, 44(5): 492-496. http://hwjs.nvir.cn/article/id/0101943c-987d-48e8-ba4d-1748f31d4bbf [14] Bavirisetti D P, Dhuli R. Two-scale image fusion of visible and infrared images using saliency detection [J]. Infrared Physics and Technology, 2016, 76: 52-64. [15] 余汪洋, 陈祥光, 董守龙, 等. 基于小波变换的图像融合算法研究[J]. 北京理工大学学报, 2014, 34(12): 1262-1266. https://www.cnki.com.cn/Article/CJFDTOTAL-BJLG201412010.htmYU W Y, CHEN X G, DONG S L, et al. Study on image fusion algorithm based on wavelet transform[J]. Transactions of Beijing Institute of Technology, 2014, 34(12): 1262-1266. https://www.cnki.com.cn/Article/CJFDTOTAL-BJLG201412010.htm [16] Redmon J, Farhadi A. YOLO9000: Better, Faster, Stronger [C]// IEEE Conference on Computer Vision & Pattern Recognition, 2017: 6517-6525.