可见光和红外图像决策级融合目标检测算法

宁大海; 郑晟

可见光和红外图像决策级融合目标检测算法

宁大海,
郑晟^,

太原理工大学电气与动力工程学院, 山西太原 030000

详细信息

作者简介:
宁大海（1998-），男，山西运城人，硕士研究生，主要从事图像处理的研究

通讯作者:
郑晟（1964-），男，山西太原人，副教授，主要从事智能控制技术和装置的研究。E-mail: 13593162661@163.com

中图分类号: TP391.41
计量
- 文章访问数: 356
- HTML全文浏览量: 113
- PDF下载量: 79
- 被引次数: 0
出版历程
- 收稿日期: 2022-08-01
- 修回日期: 2022-09-13
- 刊出日期: 2023-03-20

An Object Detection Algorithm Based on Decision-Level Fusion of Visible and Infrared Images

NING Dahai,
ZHENG Sheng^,

College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030000, China

摘要

摘要: 为了提高可见光和红外图像决策级融合目标检测算法的性能，提出了一种基于模型可靠性的决策级融合策略。首先采取图像预处理技术提高红外图像的整体质量，之后对可见光与热红外目标检测模型分别进行训练测试，根据模型测试结果得到融合策略所需参数，依据所提出的融合策略对模型检测结果进行融合，得到最后的融合检测结果。实验结果表明，相比于单一目标检测模型的检测结果，所采用的融合算法在白天的漏检率比可见光检测模型降低了8.16%，夜间漏检率比红外检测模型降低了9.85%。
- 目标检测 /
- 决策级融合 /
- 图像处理 /
- 双波段 /
- 深度学习
Abstract: To improve the performance of visible and infrared image decision-making-level fusion target detection algorithms, a decision-level fusion strategy based on model reliability was proposed. First, image preprocessing technology was adopted to improve the overall quality of an infrared image, and then visible and thermal infrared target detection models were trained and tested. The parameters required for the fusion strategy were obtained based on the model test results. The model detection results were fused according to the proposed fusion strategy, and the final fusion detection results were obtained. The experimental results showed that compared with the detection results of a single-target detection model, the missed detection rate of the fusion algorithm used in the daytime was 8.16% lower than that of the visible detection model, and the missed detection rate at night was 9.85% lower than that of the infrared detection model.
- object detection /
- decision-level fusion /
- image processing /
- dual band /
- deep learning

HTML全文

图 1 本文算法总体框架

Figure 1. General framework of our algorithm

下载: 全尺寸图片幻灯片

图 2 可见光与红外图像灰度直方图对比

Figure 2. Comparison of gray histogram between visible image and infrared image

下载: 全尺寸图片幻灯片

图 3 HE与CLAHE的效果对比图

Figure 3. Effect comparison diagram of HE and CLAHE

下载: 全尺寸图片幻灯片

图 4 引导滤波参数ε与平均梯度，PSNR的关系

Figure 4. Relationship between guiding filter parameters ε and average gradient, PSNR

下载: 全尺寸图片幻灯片

图 5 检测结果对比图

Figure 5. Comparison diagram of test results

下载: 全尺寸图片幻灯片

图 6 PP-YOLOE和其他最先进模型的比较

Figure 6. Comparison of the PP-YOLOE and other state-of-the-art models

下载: 全尺寸图片幻灯片

图 7 图像配准。左：可见光图像; 中：红外图像; 右：配准后的红外图像

Figure 7. Image registration. Left: visible image; Middle: infrared image; Right: infrared image after registration

下载: 全尺寸图片幻灯片

图 8 三个模型的训练损失曲线

Figure 8. Training loss curve of three models

下载: 全尺寸图片幻灯片

图 9 图像预处理前后模型性能对比结果

Figure 9. Comparison results of model performance before and after image preprocessing

下载: 全尺寸图片幻灯片

图 10 融合策略对比结果

Figure 10. Comparison results of fusion strategies

下载: 全尺寸图片幻灯片

图 11 检测结果可视化

Figure 11. Visualization of test results

下载: 全尺寸图片幻灯片

表 1 消融实验结果

Table 1. Ablation experiment results

Preprocessed	Primary fusion strategy	DIOU	Reliability parameter	AMR（Night）↓	AMR（Day）↓
				37.63%（Visible）	30.29%（Visible）
				30.27%（Infrared）	36.23%（Infrared）
√				27.51%（Infrared）	33.44%（Infrared）
√	√			22.14%	25.31%
√	√	√		21.06%	23.17%
√	√	√	√	20.42%	22.13%

下载: 导出CSV

参考文献(20)

[1]	LIN C, LU J, WANG G, et al. Graininess-aware deep feature learning for robust pedestrian detection[J]. IEEE Transactions on Image Processing, 2020, 29: 3820-3834. doi: 10.1109/TIP.2020.2966371
[2]	ZHANG X, YE P, LEUNG H, et al. Object fusion tracking based on visible and infrared images: a comprehensive review[J]. Information Fusion, 2020, 63: 166-187. doi: 10.1016/j.inffus.2020.05.002
[3]	唐聪, 凌永顺, 杨华, 等. 基于深度学习的红外与可见光决策级融合跟踪[J]. 激光与光电子学进展, 2019, 56(7): 217-224. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201907023.htm TANG Cong, LING Yongshun, YANG Hua, et al. Decision-Level fusion tracking for infrared and visible spectra based on deep learning[J]. Laser & Optoelectronics Progress, 2019, 56(7): 217-224. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201907023.htm
[4]	张芳芳. 红外和可见光图像融合算法的研究[D]. 洛阳: 河南科技大学, 2011. ZHANG Fangfang. The Study of Infrared and Visible Light Image Fusion Algorithm[D]. Luoyang: Henan University of Science and Technology, 2011.
[5]	Schnelle S R, Chan A L. Enhanced target tracking through infrared-visible image fusion[C]//14th International Conference on Information Fusion, IEEE, 2011: 1-8.
[6]	刘明君, 董增寿. 基于多特征的红外与可见光图像融合[J]. 激光杂志, 2019, 40(10): 81-85. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ201910017.htm LIU Mingjun, DONG Zengshou. Infrared and visible light image fusion based on multiple features[J]. Laser Journal, 2019, 40(10): 81-85. https://www.cnki.com.cn/Article/CJFDTOTAL-JGZZ201910017.htm
[7]	白玉, 侯志强, 刘晓义, 等. 基于可见光图像和红外图像决策级融合的目标检测算法[J]. 空军工程大学学报: 自然科学版, 2020, 21(6): 53-59, 100. doi: 10.3969/j.issn.1009-3516.2020.06.009 BAI Yu, HOU Zhiqiang, LIU Xiaoyi, et al. An object detection algorithm based on decision-level fusion of visible light image and infrared image[J]. Journal of Air Force Engineering University, 2020, 21(6): 53-59, 100. doi: 10.3969/j.issn.1009-3516.2020.06.009
[8]	Manpreet Kaur, Jasdeep Kaur, Jappreet Kaur. Survey of contrast enhancement techniques based on histogram equalization[J]. International Journal of Advanced Computer Science and Applications, 2011, 2(7): 137-141.
[9]	Stark J Alex. Adaptive image contrast enhancement using generalizations of histogram equalization[J]. IEEE Transactions on Image Processing, 2000, 9(5): 889-889. doi: 10.1109/83.841534
[10]	张丽. 对比度受限自适应直方图均衡方法[J]. 电脑知识与技术, 2010, 6(9): 2238-2241. doi: 10.3969/j.issn.1009-3044.2010.09.082 ZHANG Li. Contrast limited adaptive histogram equalization[J]. Computer Knowledge and Technology, 2010, 6(9): 2238-2241. doi: 10.3969/j.issn.1009-3044.2010.09.082
[11]	王加, 周永康, 李泽民, 等. 非制冷红外图像降噪算法综述[J]. 红外技术, 2021, 43(6): 557-565. http://hwjs.nvir.cn/article/id/380dcf6e-de3d-4411-ab70-e246d5c8ea27 WANG Jia, ZHOU Yongkan, LI Zemin, et al. A survey of uncooled infrared image denoising algorithms[J]. Infrared Technology, 2021, 43(6): 557-565. http://hwjs.nvir.cn/article/id/380dcf6e-de3d-4411-ab70-e246d5c8ea27
[12]	Tomasi C, Manduchi R. Bilateral filtering for gray and color images[C]//6th International Conference on Computer Vision, IEEE, 1998: 839-846.
[13]	HE K, SUN J, TANG X. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353.
[14]	唐聪, 凌永顺, 杨华, 等. 基于深度学习的红外与可见光决策级融合检测(英文)[J]. 红外与激光工程, 2019, 48(6): 456-470. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201906045.htm TANG Cong, LING Yongshun, YANG Hua, et al. Decision-level fusion detection for infrared and visible spectra based on deep learning[J]. Infrared and Laser Engineering, 2019, 48(6): 456-470. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201906045.htm
[15]	Paddle Paddle. Paddle Detection, Object detection and instance segmentation toolkit based on PaddlePaddle[EB/OL]. [2022-08-01]. https://github.com/PaddlePaddle/PaddleDetection.
[16]	XU S, WANG X, LV W, et al. PP-YOLOE: An evolved version of YOLO[J/OL]. arXiv preprint arXiv: 2203.16250, https://arxiv.org/p-df/2203.16250.pdf.
[17]	ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: Faster and better learning for bounding box regression[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12993-13000.
[18]	Hwang S, Park J, Kim N, et al. Multispectral pedestrian detection: Benchmark dataset and baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1037-1045.
[19]	LI C, SONG D, TONG R, et al. Illumination-aware faster R-CNN for robust multispectral pedestrian detection[J]. Pattern Recognition, 2019, 85: 161-171.
[20]	LI C, SONG D, TONG R, et al. Multispectral pedestrian detection via simultaneous detection and segmentation[J/OL]. arXiv preprint arXiv: 1808.04818, https://arxiv.org/pdf/1808.04818.pdf.