红外热成像中低分辨率行人小目标检测方法

胡焱; 胡皓冰; 赵宇航; 原子昊; 司成可

红外热成像中低分辨率行人小目标检测方法

中国民用航空飞行学院航空电子电气学院, 四川广汉 618307

基金项目:

国家自然科学基金项目 62061003

四川省科技计划重点研发项目 2021YFG0192

省级大学生创新创业项目 S202110624176

详细信息

作者简介:
胡焱（1973-），男，四川大英人，教授，研究生导师，研究方向：航空电子设备维修、测控。Email：huyan@cafuc.edu.cn

通讯作者:
胡皓冰（1997-），男，硕士研究生，主要从事深度学习目标检测的研究。Email：191650964@qq.com

中图分类号: TP391.4
计量
- 文章访问数: 39
- HTML全文浏览量: 14
- PDF下载量: 18
- 被引次数: 0
出版历程
- 收稿日期: 2022-04-24
- 修回日期: 2022-06-23
- 刊出日期: 2022-11-20

Infrared Thermal Imaging Low-Resolution and Small Pedestrian Target Detection Method

Institute of Avionics and Electrical Engineering, Civil Aviation Flight University of China, Guanghan 618307, China

摘要

摘要: 红外热成像图像的目标检测中，针对低分辨率小目标检测效果差、复杂尺度目标检测率低等问题，提出一种基于改进YOLOv5的红外低分辨率目标检测算法。选用LLVIP红外数据集，通过引入不同注意力机制来对比检测效果。选用效果最佳的注意力机制，改进目标检测网络的损失函数提高对小目标的检测率。利用TiX650热成像仪采集小目标图像样本对原数据集进行优化采样和增广，分别使用改进前后的YOLOv5网络进行训练。从模型训练结果和目标检测结果评估模型的性能提升，实验结果表明：相较于原始训练模型，改进后YOLOv5的训练模型，在红外成像的同一场景中对低分辨率小目标的检测精度上有明显提升，且漏检率低。
- YOLOv5 /
- 红外成像 /
- 低分辨率小目标 /
- 注意力机制
Abstract: In the target recognition of infrared thermal imaging images, a detection algorithm based on improved YOLOv5 for infrared low-resolution targets was proposed to address the poor detection of low-resolution small targets and low detection rate of complex-scale targets. The LLVIP infrared dataset was selected and the detection effect was compared by introducing different attention mechanisms. The attention mechanism with the best effect was selected to improve the loss function of the target detection network and improve the detection rate of small targets. A TiX650 thermal imager was utilized to acquire small target image samples for optimal sampling and broadening of the original dataset, and the YOLOv5 network was trained using the improved before and after, respectively. The performance improvement of the model was evaluated from the model-training and target detection results, and the experimental results demonstrate that compared with the original training model, the improved YOLOv5 training model has a significant improvement in the detection accuracy of low-resolution small targets in the same scene of infrared imaging and exhibits a low miss detection rate.
- YOLOv5 /
- infrared imaging /
- low-resolution small target /
- attention mechanism

HTML全文

图 1 实验方案设计流程图

Figure 1. Flow chart of experimental scheme design

下载: 全尺寸图片幻灯片

图 2 YOLOv5s 6.0结构

Figure 2. YOLOv5s 6.0 structure

下载: 全尺寸图片幻灯片

图 3 CIoU_Loss损失函数示意图

Figure 3. CIoU_Loss function diagram

下载: 全尺寸图片幻灯片

图 4 主干网络修改示意图

Figure 4. Schematic diagram of backbone network modification

下载: 全尺寸图片幻灯片

图 5 三种注意力机制的示意图

Figure 5. Schematic diagram of the three attention mechanisms

下载: 全尺寸图片幻灯片

图 6 两种场景下1组与8组模型的目标检测结果对比

Figure 6. Detection results of Group1 and 8 models in two scenarios

下载: 全尺寸图片幻灯片

图 7 检测数随帧数变化对比

Figure 7. Comparison of recognition number with frame rate

下载: 全尺寸图片幻灯片

表 1 红外数据集对比^[9]

Table 1. Comparison of infrared datasets

	Number of image pairs (1 frame selected per second)	Resolution	Aligned	Camera angle	Low-light	Pedestrian
TNO	261	768×576	√	shot on the ground	few	few
INO	2100	328×254	√	surveillance	√	few
OSU	285	320×240	√	surveillance	×	√
CVC-14	849	640×512	×	driving	√	√
KAIST	4750	640×480	√	driving	√	√
FILR	5258	640×512	×	driving	√	√
LLVIP	15488	1080×720	√	surveillance	√	√

下载: 导出CSV

表 2 LLVIP对比优化数据集

Table 2. Comparison LLVIP with optimized dataset

	Number of images	Resolution	Aligned	Camera angle	Low-light	Pedestrian
Original	15488	1080×720	√	surveillance	√	√
Sampling	3900	1080×720	√	surveillance	√	√
Addition	356	640×480	√	shot on the ground	√	√

下载: 导出CSV

表 3 几种目标检测算法的性能对比

Table 3. Performance comparison of several target detection algorithms

Algorithm	Infrastructure	Image Size	mAP50（VOC07+12）	mAP50(COCO)	FPS(Titan X)
Faster R-CNN	VGG-16	300×300	73.2	42.7	7
SSD300	VGG-16	300×300	74.3	41.2	46
YOLOv3	DarkNet-53	416×416	78.3	55.3	34
YOLOv5l	CSPDarknet-53	640×640	68.5	50.4	97

下载: 导出CSV

表 4 YOLOv5 6.0对比YOLOv5 5.0性能对比

Table 4. YOLOv5 6.0 vs YOLOv5 5.0 performance improvement

YOLOv5l (Large)	Size/pixels	mAP^val ^0.5:0.95	mAP^val ^0.5	Speed CPU b1/ms	Speed V100 b1/ms	Speed V00 b32/ms	Params (M)	FLOPs [@640] （B）
v5.0(previous)	640	48.2	66.9	457.9	11.6	2.8	47	115.4
v6.0(this release)	640	48.8	67.2	424.5	10.9	2.7	46.5	109.1

下载: 导出CSV

表 5 红外热成像仪主要参数

Table 5. Fluke TiX650 main parameters table

Main parameters	TiX650
Infrared resolution	640×480（307, 200 pixels）
IFOV/mrad	0.87
Field angle/°	32×24
Infrared spectral/μm	8~14
Temperature measurement range	-40℃~2000℃（-40℉~3632℉）
Accuracy	±1℃ or 1% at 25℃ ambient temperature

下载: 导出CSV

表 6 不同改进方法对平均识别精度的影响

Table 6. Effect of different improvement methods on the mAP

No.	Dataset	Replace C3 Before SPPF by			Replace non max suppression by	mAP0.5
No.	Dataset	CBAM	SE	Coordinate Attention	CIoU_nms	mAP0.5
1	Original					98.2%
2					√	98.2%
3		√				98.4%
4			√			98.4%
5				√		98.5%
6						97.2%
7	Improved				√	97.4%
8				√	√	97.6%

下载: 导出CSV

表 7 测试结果的性能对比

Table 7. Comparison of test results performance

	Numbers in scenario (a)	Numbers in scenario (b)	Average numbers per frame	Average detection rate(ms/fps)
Result 1	3	3	2.53	6
Result 2	9	11	6.08	6

下载: 导出CSV

参考文献(17)

[1]	张志强, 王萍, 于旭东, 等. 高精度红外热成像测温技术研究[J]. 仪器仪表学报, 2020, 41(5): 10-18. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB202005002.htm ZHANG Zhiqiang, WANG Ping, YU Xudong, et al. Study on high accuracy temperature measurement technology of infrared thermal imager[J]. Chinese Journal of Scientific Instrument, 2020, 41(5): 10-18. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB202005002.htm
[2]	杨其利, 周炳红, 郑伟, 等. 基于全卷积网络的红外弱小目标检测算法[J]. 红外技术, 2021, 43(4): 349-356. http://hwjs.nvir.cn/article/id/0803b37e-a6af-431f-9c4e-77c5efec85b6 YANG Qili, ZHOU Binghong, ZHENG Wei, et al. Infrared weak target detection algorithm based on full convolutional network[J]. Infrared Technology, 2021, 43(4): 349-356. http://hwjs.nvir.cn/article/id/0803b37e-a6af-431f-9c4e-77c5efec85b6
[3]	Girshick R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 1440-1448.
[4]	ZHAO M, CHENG L, YANG X, et al. TBC-Net: A real-time detector for infrared small target detection using semantic constraint[J/OL]. arXiv preprint arXiv: 2001.05852, 2019.
[5]	ZHAO B, WANG C, FU Q, et al. A novel pattern for infrared small target detection with generative adversarial network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(5): 4481-4492.
[6]	LIU W, Anguelov D, Erhan D, et al. SSD: Single shot multibox detector[C]//European Conference on Computer Vision, 2016: 21-37.
[7]	Redmon J, Farhadi A. Yolov3: an incremental improvement[J/OL]. Computer Vision and Pattern Recognition, https://arxiv.org/abs/1804.02767.
[8]	SUN Y, SHAO Y, YANG G, et al. A Method of Infrared Image Pedestrian Detection with Improved YOLOv3 Algorithm[J]. American Journal of Optics and Photonics, 2021, 9(3): 32-38. doi: 10.11648/j.ajop.20210903.11
[9]	邱天衡, 王玲, 王鹏, 等. 基于改进YOLOv5的目标检测算法研究[J]. 计算机工程与应用, 2022, 58(13): 63-73. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202213006.htm QIU Tianheng, WANG Ling, WANG Peng, et al. Research on object detection algorithm based on improved YOLOv5[J]. Computer Engineering and Applications, 2022, 58(13): 63-73. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202213006.htm
[10]	郭磊, 王邱龙, 薛伟, 等. 基于改进YOLOv5的小目标检测算法[J]. 电子科技大学学报, 2022, 51(2): 251-258. https://www.cnki.com.cn/Article/CJFDTOTAL-DKDX202202013.htm GUO Lei, WANG Qiulong, XUE Wei, et al. A Small Object Detection Algorithm Based on Improved YOLOv5[J]. Journal of University of Electronic Science and Technology of China, 2022, 51(2): 251-258. https://www.cnki.com.cn/Article/CJFDTOTAL-DKDX202202013.htm
[11]	JIA X, ZHU C, LI M, et al. LLVIP: A visible-infrared paired dataset for low-light vision[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 3496-3504.
[12]	TAN S, YAN J, JIANG Z, et al. Approach for improving YOLOv5 network with application to remote sensing target detection[J]. Journal of Applied Remote Sensing, 2021, 15(3): 036512.
[13]	REN S, HE K, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149
[14]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[15]	Woo S, Park J, Lee J Y, et al. CBAM: Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.
[16]	HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13713-13722.
[17]	聂鹏, 肖欢, 喻聪. YOLOv5预测边界框分簇自适应损失权重改进模型[J/OL]. 控制与决策, [2022-02-07], https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CAPJ&dbname=CAPJLAST&filename=KZYC2022020100G&uniplatform=NZKPT&v=SFfl7Znyzm6DrlyluGWpdKIbirJm5mFkFaafF4bE9xywSjZRggaM-XJWOO0if82l. NIE Peng, XIAO Huan, YU Cong. Enhanced self-adaptive loss weight YOLOv5 model based on predicted[J/OL]. Control and Decision, [2022-02-07], https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CAPJ&dbname=CAPJLAST&filename=KZYC2022020100G&uniplatform=NZKPT&v=SFfl7Znyzm6DrlyluGWpdKIbirJm5mFkFaafF4bE9xywSjZRggaM-XJWOO0if82l.