基于偏振图像的低照度场景多目标检测算法

寻华生; 张晶晶; 刘晓; 李腾; 年福东; 张馨

基于偏振图像的低照度场景多目标检测算法

寻华生^{1, 2,},
张晶晶^{1, 2, 3, ,},
刘晓³,
李腾^{1, 2},
年福东^{1, 4},
张馨^{1, 2}

1.
安徽大学计算智能与信号处理教育部重点实验室, 安徽合肥 230601
2.
偏振光成像探测技术安徽省重点实验室, 安徽合肥 230031
3.
中国科学院通用光学定标与表征技术重点实验室, 安徽合肥 230031
4.
合肥学院先进制造工程学院, 安徽合肥 230601

基金项目:

国家自然科学基金青年科学基金项目 61902104

安徽省自然科学基金项目 2008085QF295

安徽高校自然科学研究项目 KJ2020A0651

安徽省自然科学基金项目 1808085MF

详细信息

作者简介:
寻华生（1996-），男，硕士研究生，主要研究方向为图像处理、深度学习。E-mail: Z19201027@stu.ahu.edu.cn

通讯作者:
张晶晶（1974-），女，副教授，博士，主要研究方向为图像处理、遥感信息处理和模式识别。E-mail: fannyzjj@ahu.edu.cn

中图分类号: TP183
计量
- 文章访问数: 266
- HTML全文浏览量: 74
- PDF下载量: 69
- 被引次数: 0
出版历程
- 收稿日期: 2021-06-15
- 修回日期: 2021-08-03
- 刊出日期: 2022-05-20

Multi-Target Detection of Low-Illuminance Scene Based on Polarization Image

XUN Huasheng^{1, 2
,},
ZHANG Jingjing^{1, 2, 3
, ,},
LIU Xiao³,
LI Teng^{1, 2},
NIAN Fudong^{1, 4},
ZHANG Xin^{1, 2}

1.
Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education, Anhui University, Hefei 230601, China
2.
Anhui Province Key Laboratory of Polarization Detection Technology, Hefei 230031, China
3.
Key Laboratory of Optical Calibration and Characterization, Chinese Academy of Sciences, Hefei 230031, China
4.
School of Advanced Manufacturing Engineering, Hefei University, Hefei 230601, China

摘要

摘要: 偏振光反射信息可直接反演目标本征特性，且在传输过程中具备较强的抗干扰特性，因此偏振成像技术可适用于多种复杂环境中的智能监控、交通监察领域。近年来使用深度学习判读图像检测目标的方法迅速发展，已经广泛应用于图像处理的各个领域。本文提出了一种基于偏振图像与深度神经网络算法的行人、车辆多目标检测算法YOLOv5s-DOLP。首先，通过实时获取到偏振图像进行偏振信息解析，获取目标偏振度图像。其次，为增强偏振度图像中检测目标与背景存在高对比度的特性，在主干网络中引入通道注意力与空间注意力，提升网络特征进行自适应学习的能力。此外，使用K-means算法对目标位置信息进行聚类分析，加快网络在偏振度图像的学习速度，提升目标检测精度。实验结果显示，该算法结合了偏振成像和深度学习目标检测的优势，对于低照度复杂场景中的车辆、行人目标检测效果好、检测速度快，对于道路车辆的目标检测、识别与跟踪具有一定的应用价值。
- 偏振成像 /
- 神经网络 /
- YOLOv5s /
- 多目标检测 /
- 注意力机制
Abstract: Polarized light reflection information can directly invert the intrinsic characteristics of a target and has strong anti-interference characteristics in the transmission process. Thus, polarization imaging technology can be applied to the fields of intelligent monitoring and traffic monitoring in various complex environments. In recent years, deep-neural-network methods for interpreting image detection targets have been developed rapidly and widely used in various fields of image processing. In this study, a vehicle multi-target detection algorithm based on polarized images and deep learning is proposed. First, the target polarization degree image can be obtained by acquiring the polarization image in real time and analyzing the polarization information. Second, to enhance the high contrast between the detection targets and the background in the polarization image, channel attention and spatial attention are introduced into the backbone network to improve the ability of the network features to perform adaptive learning. In addition, the K-means algorithm is used to perform clustering analysis on the target location information, thereby increasing the network's learning speed in the polarization image and improving the progress of target detection. The experimental results show that this method is effective and fast for vehicle detection in complex scenes with low illumination. This method combines the advantages of polarization imaging and deep-learning target detection and has substantial application scope in road vehicle target detection, recognition, and tracking.
- polarization image /
- neural network /
- YOLO v5s /
- multi-target detection /
- attention mechanism

HTML全文

图 1 偏振图像采集与偏振参量解析流程

Figure 1. Polarization image acquisition and polarization parameter analysis process

下载: 全尺寸图片幻灯片

图 2 偏振参量解析示意图

Figure 2. Process of polarization parameter analysis

下载: 全尺寸图片幻灯片

图 3 算法流程图

Figure 3. Algorithm flowchart

下载: 全尺寸图片幻灯片

图 4 Focus操作示意图

Figure 4. Focus operation diagram

下载: 全尺寸图片幻灯片

图 5 拉普拉斯算子^[15]检测结果

Figure 5. Detecting result of Laplace

下载: 全尺寸图片幻灯片

图 6 卷积块注意力模块

Figure 6. Convolutional block attention module

下载: 全尺寸图片幻灯片

图 7 通道注意力模块

Figure 7. Channel attention module

下载: 全尺寸图片幻灯片

图 8 空间注意力模块

Figure 8. Spatial attention module

下载: 全尺寸图片幻灯片

图 9 YOLO v5s-DOLP网络

Figure 9. YOLO v5s-DOLP network

下载: 全尺寸图片幻灯片

图 10 检测结果示例

Figure 10. Detect results of samples

下载: 全尺寸图片幻灯片

图 11 Precision-Recall曲线

Figure 11. Precision-Recall curves

下载: 全尺寸图片幻灯片

图 12 消融实验检测结果示例

Figure 12. Detect results of samples of ablation experiment

下载: 全尺寸图片幻灯片

表 1 分光型偏振成像设备

Table 1. Spectroscopic polarization imaging equipment

Max resolution	Frame	Image sensor	Pixel size/μm	ADC
2488×2048	65	Sony IMX250，CMOS，2/3 Global shutter	3.45	10 bit/12 bit

下载: 导出CSV

表 2 网络训练环境

Table 2. Network training environment

Name	Configure
CPU	Intel Xeon E5-2630
GPU	NVIDIA 1080Ti * 2
Operating system	Ubuntu 18.04
Parallel computing library	Cuda10 + Cudnn7.4
Image processing	Python3.6、Opencv3.4.0
Deep learning framework	Pytorch

下载: 导出CSV

表 3 目标检测算法检测结果

Table 3. Detect results of target detection algorithm

Methods	AP		mAP	Test time/s
Methods	Person	Car	mAP	Test time/s
Faster R-CNN(Res 50)	91.6	97.5	94.6	0.103
YOLOv4(Res 50)	91.6	97.3	94.5	0.029
YOLOv5s-DOLP	94.8	98.9	96.9	0.039

下载: 导出CSV

表 4 消融实验结果

Table 4. Results of ablation experiment

Methods	AP		mAP	Test time/s
Methods	Person	Car	mAP	Test time/s
YOLOv5s	87.2	98.3	92.8	0.027
A	90.2	98.5	94.3	0.029
B	92.0	98.8	95.4	0.034
C	88.5	98.4	93.4	0.027

下载: 导出CSV

参考文献(21)

[1]	于洵, 杨烨, 姜旭, 等. 基于偏振光谱成像的目标识别方法研究[J]. 应用光学, 2016, 37(4): 537-541. https://www.cnki.com.cn/Article/CJFDTOTAL-YYGX201604008.htm YU Xun, YANG Ye, JIANG Xu, et al. Recognition of camouflage targets by polarization spectral imaging system[J]. Journal of Applied Optics, 2016, 37(4): 537-541. https://www.cnki.com.cn/Article/CJFDTOTAL-YYGX201604008.htm
[2]	李从利, 薛松, 陆文骏, 等. 雾天条件下偏振解析成像质量评价[J]. 中国图象图形学报, 2017, 22(3): 366-375. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201703011.htm LI Congli, XUE Song, LU Wenjun, et al. Quality assessment of polarization imaging under foggy[J]. Journal of Image and Graphics, 2017, 22(3): 366-375. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB201703011.htm
[3]	李小明, 黄勤超. 沙漠背景下红外偏振图像目标检测方法[J]. 红外技术, 2016, 38(9): 779-782, 792. http://hwjs.nvir.cn/article/id/hwjs201609012 LI Xiaoming, HUANG Qinchao. Target detection for infrared polarization image in the background of desert[J]. Infrared Technology, 2016, 38(9): 779-782, 792. http://hwjs.nvir.cn/article/id/hwjs201609012
[4]	张荣, 李伟平, 莫同. 深度学习研究综述[J]. 信息与控制, 2018, 47(4): 385-397. https://www.cnki.com.cn/Article/CJFDTOTAL-XXYK201804002.htm ZHANG Rong, LI Weiping, MO Tong. Review of deep learning[J]. Information and Control, 2018, 47(4): 385-397. https://www.cnki.com.cn/Article/CJFDTOTAL-XXYK201804002.htm
[5]	王文秀, 傅雨田, 董峰, 等. 基于深度卷积神经网络的红外船只目标检测方法[J]. 光学学报, 2018, 38(7): 0712006. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201807020.htm WANG Wenxiu, FU Yutian, DONG Feng, et al. Infrared ship target detection method based on deep convolution neural network[J]. Acta Optica Sinica, 2018, 38(7): 0712006. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201807020.htm
[6]	罗海波, 许凌云, 惠斌, 等. 基于深度学习的目标跟踪方法研究现状与展望[J]. 红外与激光工程, 2017, 46(5): 0502002-0502002(7). https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201705002.htm LUO Haibo, XU Lingyun, HUI Bin, et al. Status and prospect of target tracking based on deep learning[J]. Infrared and Laser Engineering, 2017, 46(5): 0502002-0502002(7). https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201705002.htm
[7]	Ross Girshick. Fast R-CNN[J]. Computer Science, 2015, 6: 1440-1448.
[8]	REN Shaoqing, HE Kaiming, Ross Girshick, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149.
[9]	HE Kaiming, Georgia Gkioxari, Piotr Dollar, et al. Mask R-CNN[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision, 2017: 2961-2969.
[10]	LIU Wei, Dragomir Anguelov, Dumitru Erhan, et al. SSD: Single shot multibox detector[C]//European Conference on Computer Vision, 2016, 6: 21-27.
[11]	Joseph Redmon, Santosh Divvala, Ross Girshick, et al. You Only Look Once: unified, real-time object detection[C]//Conference on Computer Vision and Pattern Recognition, 2016, 6: 779-788.
[12]	Joseph Redmon, Ali Farhadi. YOLO9000: better, faster, stronger[C]// Conference on Computer Vision and Pattern Recognition, 2017, 7: 6517-6525
[13]	Joseph Redmon, Ali Farhadi. YOLOv3: an incremental improvement [J/OL]. arXiv: 1804.02767, 2018.
[14]	Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao. YOLOv4: optimal speed and accuracy of object detection[J/OL]. arXiv: 2004. 10934, 2020.
[15]	冈萨雷斯. 数字图像处理[M]. 第三版, 北京: 电子工业出版社, 2011: 26-29. Rafael C. Gonzalez. Digital Image Processing[M]. 3th, Beijing: Publishing House of Electronics Industry, 2011: 26-29.
[16]	Woo S, Park J, Lee J Y, et al. Cbam: Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 3-19.
[17]	张汝榛, 张建林, 祁小平, 等. 复杂场景下的红外目标检测[J]. 光电工程, 2020, 47(10): 200314. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202010010.htm ZHANG R Z, ZHANG J L, QI X P, et al. Infrared target detection and recognition in complex scene[J]. Opto-Electron Eng., 2020, 47(10): 200314. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202010010.htm
[18]	宫剑, 吕俊伟, 刘亮, 等. 红外偏振图像的舰船目标检测[J]. 光谱学与光谱分析, 2020, 40(2): 586-594. https://www.cnki.com.cn/Article/CJFDTOTAL-GUAN202002052.htm GONG Jian, LYU Junwei, LIU Liang, et al. Ship target detection based on infrared polarization image[J]. Spectroscopy and Spectral Analysis, 2020, 40(2): 586-594. https://www.cnki.com.cn/Article/CJFDTOTAL-GUAN202002052.htm
[19]	游江, 刘鹏祖, 容晓龙, 等. 基于暗通道先验原理的偏振图像去雾增强算法研究[J]. 激光与红外, 2020, 50(4): 493-500. doi: 10.3969/j.issn.1001-5078.2020.04.019 YOU Jiang, LIU Pengzu, RONG Xiaolong, et al. Dehazing and enhancement research of polarized image based on dark channel priori principle[J]. Laser & Infrared, 2020, 50(4): 493-500. doi: 10.3969/j.issn.1001-5078.2020.04.019
[20]	王美荣, 徐国明, 袁宏武. 显著性偏振参量深度稀疏特征学习的目标检测方法[J]. 激光与光电子学进展, 2019, 56(19): 191101. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201919013.htm WANG Meirong, XU Guoming, YUAN Hongwu. Object detection by deep sparse feature learning of salient polarization parameters[J]. Laser & Optoelectronics Progress, 2019, 56(19): 191101. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201919013.htm
[21]	李慕锴, 张涛, 崔文楠. 基于YOLOv3的红外行人小目标检测技术研究[J]. 红外技术, 2020, 42(2): 176-181. http://hwjs.nvir.cn/article/id/hwjs202002012 LI Mukai, ZHANG Tao, CUI Wennan. Research of infrared small pedestrian target detection based on YOLOv3[J]. Infrared Technology, 2020, 42(2): 176-181. http://hwjs.nvir.cn/article/id/hwjs202002012