Advanced Search
中文
Volume 45 Issue 12
Dec.  2023
Turn off MathJax
Article Contents
Abstract
References
Related Articles
CHEN Xu, WU Wei, PENG Dongliang, GU Yu. Infrared-PV: an Infrared Target Detection Dataset for Surveillance Application[J]. Infrared Technology , 2023, 45(12): 1304-1313.
Citation: CHEN Xu, WU Wei, PENG Dongliang, GU Yu. Infrared-PV: an Infrared Target Detection Dataset for Surveillance Application[J]. Infrared Technology , 2023, 45(12): 1304-1313.
shu

Infrared-PV: an Infrared Target Detection Dataset for Surveillance Application

  • 1.

    School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China

  • 2.

    No. 28 Research Institute of China Electronic Technology Group, Nanjing 210007, China

More Information
  • Received Date: January 14, 2021
  • Revised Date: February 23, 2021
    Graphical Abstract
    • Abstract
  • Although infrared cameras can operate day and night under all-weather conditions compared with visible cameras, the infrared images obtained by them have low resolution and signal-to-clutter ratio, lack of texture information, so enough labeled images and optimization model design have great influence on improving infrared target detection performance based on deep learning. First, to solve the lack of an infrared target detection dataset used for surveillance applications, an infrared camera was used to capture images with multiple polarities, and an image annotation task that outputted the VOC format was performed using our developed annotation software. An infrared image dataset containing two types of targets, person and vehicle, was constructed and named infrared-PV. The characteristics of the targets in this dataset were statistically analyzed. Second, state-of-the-art target detection models based on deep learning were adopted to perform model training and testing. Target detection performances for this dataset were qualitatively and quantitatively analyzed for the YOLO and Faster R-CNN series detection models. The constructed infrared dataset contained 2138 images, and the targets in this dataset included three types of modes: white hot, black hot, and heat map. In the benchmark test using several models, Cascade R-CNN achieves the best performance, where mean average precision when intersection over union exceeding 0.5 (mAP0.5) reaches 82.3%, and YOLOv5 model can achieve the tradeoff between real-time performance and detection performance, where inference time achieves 175.4 frames per second and mAP0.5 drops only 2.7%. The constructed infrared target detection dataset can provide data support for research on infrared image target detection model optimization and can also be used to analyze infrared target characteristics.
    • FullText(HTML)
    • References (40)
  • [1]
    陈钱, 隋修宝. 红外图像处理理论与技术[M]. 北京: 电子工业出版社, 2018.

    CHEN Qian, SUI Xiubao. Infrared Image Processing Theory and Technology[M]. Beijing: Electronic Industry Press, 2018.
    [2]
    刘让, 王德江, 贾平, 等. 红外图像弱小目标探测技术综述[J]. 激光与光电子学进展, 2016, 53(5): 050004. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201605004.htm

    LIU Rang, WANG Dejiang, JIA Ping, et al. Overview of detection technology for small and dim targets in infrared images[J]. Progress in Laser and Optoelectronics, 2016, 53(5): 050004. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201605004.htm
    [3]
    武斌. 红外弱小目标检测技术研究[D]. 西安: 西安电子科技大学. 2009.

    WU Bing. Research on Infrared Dim Target Detection Technology[D]. Xi'an: Xidian University, 2009.
    [4]
    Rawat S S, Verma S K, Kumar Y. Review on recent development in infrared small target detection algorithms[J]. Procedia Computer Science, 2020, 167: 2496-2505. DOI: 10.1016/j.procs.2020.03.302
    [5]
    李俊宏, 张萍, 王晓玮, 等. 红外弱小目标检测算法综述[J]. 中国图象图形学报, 2020, 25(9): 1739-1753. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202009002.htm

    LI Junhong, ZHANG Ping, WANG Xiaowei, et al. Infrared small-target detection algorithms: a survey[J]. Journal of Image and Graphics, 2020, 25(9): 1739-1753. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202009002.htm
    [6]
    谷雨, 刘俊, 沈宏海, 等. 基于改进多尺度分形特征的红外图像弱小目标检测[J]. 光学精密工程, 2020, 28(6): 1375-1386. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM202006016.htm

    GU Yu, LIU Jun, SHEN Honghai, et al. Infrared image dim target detection based on improved multi-scale fractal features[J]. Optics and Precision Engineering, 2020, 28(6): 1375-1386. https://www.cnki.com.cn/Article/CJFDTOTAL-GXJM202006016.htm
    [7]
    LIU L, OUYANG W, WANG X G, et al. Deep learning for generic object detection: a survey[J]. International Journal of Computer Vision, 2020, 128(2): 261-318. DOI: 10.1007/s11263-019-01247-4
    [8]
    REN S Q, HE K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI: 10.1109/TPAMI.2016.2577031
    [9]
    LIU W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector[C]//European Conference on Computer Vision, 2016: 21-37.
    [10]
    Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779-788.
    [11]
    王文秀, 傅雨田, 董峰, 等. 基于深度卷积神经网络的红外船只目标检测方法[J]. 光学学报, 2018, 38(7): 0712006. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201807020.htm

    WANG W X, FU Y T, DONG F, et al. Infrared ship target detection method based on deep convolutional neural network[J]. Acta Optics, 2018, 38(7): 0712006. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201807020.htm
    [12]
    Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. DOI: 10.1145/3065386
    [13]
    蒋志新. 基于深度学习的海上红外小目标检测方法研究[D]. 大连: 大连海事大学, 2019.

    JIANG Z X. Research on the Detection Method of Marine Infrared Small Target Based on Deep Learning[D]. Dalian: Dalian Maritime University, 2019.
    [14]
    陈铁明, 付光远, 李诗怡, 等. 基于YOLO v3的红外末制导典型目标检测[J]. 激光与光电子学进展, 2019, 56(16): 155-162. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202222030.htm

    CHEN T M, FU G Y, LI S Y, et al. Infrared terminal guidance typical target detection based on YOLOv3[J]. Progress in Laser and Optoelectronics, 2019, 56(16): 155-162. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202222030.htm
    [15]
    赵琰, 刘荻, 赵凌君. 基于Yolo v3的复杂环境红外弱小目标检测[J]. 航空兵器, 2020, 26(6): 29-34. https://www.cnki.com.cn/Article/CJFDTOTAL-HKBQ201906006.htm

    ZHAO Y, LIU D, ZHAO L J. Infrared small target detection in complex environment based on Yolo v3[J]. Aviation Weaponry, 2020, 26(6): 29-34. https://www.cnki.com.cn/Article/CJFDTOTAL-HKBQ201906006.htm
    [16]
    吴双忱, 左峥嵘. 基于深度卷积神经网络的红外小目标检测[J]. 红外与毫米波学报, 2019, 38(3): 371-380. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYH201903019.htm

    WU S C, ZUO Z G. Infrared small target detection based on deep convolutional neural network[J]. Journal of Infrared and Millimeter Waves, 2019, 38(3): 371-380. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYH201903019.htm
    [17]
    HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
    [18]
    李慕锴, 张涛, 崔文楠. 基于Yolo v3的红外行人小目标检测技术研究[J]. 红外技术, 2020, 42(2): 176-181. http://hwjs.nvir.cn/article/id/hwjs202002012

    LI M K, ZHANG T, CUI W N. Research on infrared pedestrian small target detection technology based on Yolo v3[J]. Infrared Technology, 2020, 42(2): 176-181. http://hwjs.nvir.cn/article/id/hwjs202002012
    [19]
    Everingham M, Eslami S A, Van Gool L, et al. The pascal visual object classes challenge: a retrospective[J]. International Journal of Computer Vision, 2015, 111(1): 98-136. DOI: 10.1007/s11263-014-0733-5
    [20]
    LIN T Y, Maire M, Belongie S, et al. Microsoft coco: common objects in context[C]//European Conference on Computer Vision, 2014: 740-755.
    [21]
    XIA G S, BAI X, DING J, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3974-3983.
    [22]
    LI K, WAN G, CHENG G, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 296-307. DOI: 10.1016/j.isprsjprs.2019.11.023
    [23]
    ZHU H, CHEN X, DAI W, et al. Orientation robust object detection in aerial images using deep convolutional neural network[C]//2015 IEEE International Conference on Image Processing (ICIP), 2015: 3735-3739.
    [24]
    TAN M, PANG R, LE Q V. Efficientdet: scalable and efficient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10781-10790.
    [25]
    Hwang S, Park J, Kim N, et al. Multispectral pedestrian detection: Benchmark dataset and baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1037-1045.
    [26]
    Teledyne FLIR Systems. FLIR ADAS Dataset[DB/OL] [2023-11-27]. https://www.flir.com/oem/adas/adas-dataset-form/.
    [27]
    Davis J W, Keck M A. A two-stage template approach to person detection in thermal imagery[C]//2005 Seventh IEEE Workshops on Applications of Computer Vision, 2005, 1: 364-369.
    [28]
    CAI Z, Vasconcelos N. Cascade r-cnn: delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 6154-6162.
    [29]
    PANG J, CHEN K, SHI J, et al. Libra r-cnn: Towards balanced learning for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 821-830.
    [30]
    WU Y, CHEN Y, YUAN L, et al. Rethinking classification and localization for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10186-10195.
    [31]
    Redmon J, Farhadi A. Yolov3: an incremental improvement [EB/OL] [2018-04-08]. https://arxiv.org/pdf/1804.02767.pdf.
    [32]
    HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
    [33]
    LINT Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
    [34]
    Bochkovskiy A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[EB/OL] [2020-04-22]. https://arxiv.org/pdf/2004.10934.pdf.
    [35]
    HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
    [36]
    WANG K, LIEW J H, ZOU Y, et al. PaNet: Few-shot image semantic segmentation with prototype alignment[C]//Proceedings of the IEEE International Conference on Computer Vision, 2019: 9197-9206.
    [37]
    ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of 2020 Association for the Advancement of Artificial Intelligence, 2020: 12993-13000.
    [38]
    TIAN Z, SHEN C, CHEN H, et al. FCOS: Fully convolutional one-stage object detection[C]//Proceedings of the 2019 IEEE International Conference on Computer Vision, 2019: 9627-9636.
    [39]
    CHEN K, WANG J Q, PANG J M, et al. Mmdetection: open mmlab detection toolbox and benchmark[EB/OL][2019-06-17]. https://arxiv.org/pdf/1906.07155.pdf.
    [40]
    ZHANG H, WU C R, ZHANG Z Y, et al. Resnest: Split-attention networks[EB/OL] [2020-04-19]. https://arxiv.org/pdf/2004.08955.pdf.
    • Related Articles

  • Related Articles

    [1]PAN Hao, MA Yi, ZHOU Fangrong, MA Yutang, QIAN Guochao, WEN Gang. Research on the Theoretical Model Between Solar-blind UV and Atmospheric Temperature during Atmospheric Transmission[J]. Infrared Technology , 2020, 42(10): 1007-1012.
    [2]CAO Hongye, ZHANG Tianqi. Atmospheric Correction Algorithm for GF-2 Image Based On a Radiative Transfer Model[J]. Infrared Technology , 2020, 42(6): 534-541.
    [3]FU Xiaoning, CHEN Liqiang, DONG Que. Passive Ranging Algorithm Based on Improved Elman Neural Network[J]. Infrared Technology , 2019, 41(6): 540-544.
    [4]LIU Bingqi, YU Hao, YAN Zongqun, ZHANG Yu, CHEN Yichao, ZHANG Shuai. A Fast Calculation Model for Average Transmission of Oxygen A Absorption Band in Homogeneous Atmosphere[J]. Infrared Technology , 2018, 40(11): 1056-1060.
    [5]CHEN Fang-fang, GENG Rui, LYU Yong. Research on the Transmittance Model of Laser Infrared Atmospheric Transmission[J]. Infrared Technology , 2015, (6): 496-501.
    [6]ZHAO Gang, ZHANG Kai, SHAO Wei, YAN Jie. An Infrared Degradation Image Simulation Based on High-speed Turbulent Statistical Model[J]. Infrared Technology , 2014, (4): 294-297.
    [7]ZHANG Ning, WU He-ran, ZHOU Yun, JIANG Ning, JIANG Ya-dong. Using the Three-dimension Noise Model to Test and Analyze the ROIC of UIRFPA[J]. Infrared Technology , 2012, 34(6): 336-339. DOI: 10.3969/j.issn.1001-8891.2012.06.006
    [8]GAO Wen-guang, SUN Ji-yin, LIU Hao. The Simulation Models of Atmospheric Infrared Emissinon Based on Database[J]. Infrared Technology , 2010, 32(6): 333-336. DOI: 10.3969/j.issn.1001-8891.2010.06.006
    [9]LI Jun-wei, ZHAN Guo-zhong, GUO Fang-min. An Equivalent Circuit Model and Its Readout IC Design of a Quantum Dot-Quantum Well Hybrid Structure Optical Memory Cell[J]. Infrared Technology , 2009, 31(7): 386-389. DOI: 10.3969/j.issn.1001-8891.2009.07.004
    [10]A More Reliable Image Information Processing System Based on Data-Fusion Model[J]. Infrared Technology , 2004, 26(6): 17-21. DOI: 10.3969/j.issn.1001-8891.2004.06.005

  • Cited by

    Periodical cited type(1)

    1. 王东，高俊光，陈磊，张恒伟，路亚旭. 喷气式飞机尾喷管及尾焰红外辐射特性测量分析. 激光与红外. 2024(07): 1097-1101 .

    Other cited types(1)

Catalog

    Get Citation
    PDF
    XML
    Article views (270) PDF downloads (58) Cited by(2)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return