Advanced Search
中文
Volume 43 Issue 9
Sep.  2021
Turn off MathJax
Article Contents
Abstract
References
Related Articles
JIANG Yi, HOU Liping, ZHANG Qiang. Infrared Pedestrian Action Recognition Based on Improved Spatial-temporal Two-stream Convolution Network[J]. Infrared Technology , 2021, 43(9): 852-860.
Citation: JIANG Yi, HOU Liping, ZHANG Qiang. Infrared Pedestrian Action Recognition Based on Improved Spatial-temporal Two-stream Convolution Network[J]. Infrared Technology , 2021, 43(9): 852-860.
shu

Infrared Pedestrian Action Recognition Based on Improved Spatial-temporal Two-stream Convolution Network

  • 1.

    School of Mathematics and Computer Science, Xinyang Polytechnic College, Xinyang 464000, China

  • 2.

    School of Information Engineering, Xinyang Agriculture and Forestry College, Xinyang 464000, China

  • 3.

    Graduate Team, the First Aviation Academy, Chinese Air force, Xinyang 464000, China

More Information
  • Received Date: December 26, 2020
  • Revised Date: August 23, 2021
    Graphical Abstract
    • Abstract
  • This study proposes an improved spatial-temporal two-stream network to improve the pedestrian action recognition accuracy of infrared sequences in complex backgrounds. First, a deep differential network replaces the temporal stream network to improve the representation ability and extraction efficiency of spatio-temporal features. Then, the improved softmax loss function based on the decision-making level feature fusion mechanism is used to train the model, which can retain the spatio-temporal characteristics of images between different network frames to a greater extent and reflect the action category of pedestrians more realistically. Simulation results show that the proposed improved network achieves 87% recognition accuracy on the self-built infrared dataset, and the computational efficiency is improved by 25%, which has a high engineering application value.
    • FullText(HTML)
    • References (24)
  • [1]
    Karpathy A, Toderici G, Shetty S, et al. Large- scale video classification with convolutional neural networks[C]// CVPR, 2014: 1725-1732.
    [2]
    Tran D, Bourdev L D, Fergus R, et al. Learning spatiotem-poral features with 3d convolutional networks[C]//ICCV, 2015: 4489-4497.
    [3]
    ZHANG B, WANG L, WANG Z, et al. Real-time action recognition with enhanced motion vector CNNs[C]//CVPR, 2016: 2718-2726.
    [4]
    Niebles J C, CHEN C W, LI F F. Modeling temporal structure of decomposable motion segments for activity classification[C]// ECCV, 2010: 392-405.
    [5]
    Tumas P, Nowosielski A, Serackis A. Pedestrian detection in severe weather conditions[J]. IEEE Access, 2020, 8: 62775-62784. DOI: 10.1109/ACCESS.2020.2982539
    [6]
    魏丽, 丁萌, 曾丽君. 红外图像中基于似物性与稀疏编码的行人检测[J]. 红外技术, 2016, 38(9): 752-757. http://hwjs.nvir.cn/article/id/hwjs201609007

    WEI Li, DING Meng, ZENG Lijun. Pedestrian Detection Based on Objectness and Sparse Coding in a Single Infrared Image[J]. Infrared Technology, 2016, 38(9): 752-757. http://hwjs.nvir.cn/article/id/hwjs201609007
    [7]
    Fernando B, Gavves E M, Ghodrati J O, et al. Modeling video evolution for action recognition[C]//CVPR, 2015: 5378-5387.
    [8]
    Varol G, Laptev I, Schmid C. Long-term temporal convolutions for action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 40(6): 1510-1517.
    [9]
    Donahue J, Anne Hendricks L, Guadarrama S, et al. Long-term Recurrent Convolutional Networks for Visual Recognition and Description[M]. Elsevier, 2015: 2625-2634.
    [10]
    Soomro K, Zamir A R, Shah M. A dataset of 101 human actions classes from videos in the wild[J/OL]. Computer Vision and Pattern Recognition, arXiv: 1212.0402, 2012.
    [11]
    Kuehne H, Jhuang H, Garrote E, et al. HMDB: A large video database for human motion recognition[C]//ICCV, 2011: 2556-2563.
    [12]
    Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]//ICML, 2015: 448-456.
    [13]
    WANG L, QIAO Y, TANG X. Video action detection with relational dynamic- poselets[C]//ECCV, 2014: 565-580.
    [14]
    GAN C, YAO T, YANG K, et a. You lead, we exceed: Labor-free video concept learning by jointly exploiting web videos and images[C]//CVPR, 2016: 923-932.
    [15]
    Simonyan K, Zisserman A. Two-Stream Convolutional Networks for Action Recognition in Videos[J]. Advances in Neural Information Processing Systems, 2014, 150: 109-125. http://de.arxiv.org/pdf/1406.2199
    [16]
    冉鹏, 王灵, 李昕, 等. 改进Softmax分类器的深度卷积神经网络及其在人脸识别中的应用[J]. 上海大学学报: 自然科学版, 2018, 24(3): 352-366. https://www.cnki.com.cn/Article/CJFDTOTAL-SDXZ201803004.htm

    RAN Peng, WANG Ling, LI Xin, et al. Deep convolution neural network based on improved softmax classifier and its application in face recognition[J]. Journal of Shanghai University: Natural Science Edition, 2018, 24(3): 352-366. https://www.cnki.com.cn/Article/CJFDTOTAL-SDXZ201803004.htm
    [17]
    Yasin H, Hussain M, Weber A. Keys for Action: An Efficient Keyframe-Based Approach for 3D Action Recognition Using a Deep Neural Network[J]. Sensors, 2020, 20(8): 2226. DOI: 10.3390/s20082226
    [18]
    GAO Chenqiang, DU Yinhe, LIU Jiang, et al. InfAR dataset: Infrared action recognition at different times[J]. Neurcomputing, 2016, 212: 36-47. DOI: 10.1016/j.neucom.2016.05.094
    [19]
    WANG H, SCHMID C. Action recognition with improved trajectories[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2013: 3551-3558.
    [20]
    Du Tran, Lubomir Bourdev, Rob Fergus, et al. Learning spatiotemporal features with 3D convolutional networks[C]//Proceedings of the 2015 IEEE, International Conference on Computer Vision. Piscataway: IEEE, 2015: 4489-4497.
    [21]
    杨天明, 陈志, 岳文静. 基于视频深度学习的时空双流人物动作识别模型[J]. 计算机应用, 2018, 38(3): 895-899. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY201803050.htm

    YANG T M, CHENG Z, YU, W J, et al. Spatio-temporal two-stream human action recognition model based on video deep learning[J]. Journal of Computer Applications, 2018, 38(3): 895-899, 915. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJY201803050.htm
    [22]
    LIN S, JIA K, CHEN K, et al. Lattice long short-term memory for human action recognition[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2166-2175.
    [23]
    Carrlira J, Gisslrman A. Quo vadis. action recognition? A new model and the kinetics dataset[C]//Proceedings of the 2017 IEEE, Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 4724-4733.
    [24]
    SUN S, KUANG Z, SHENG L, et al. Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018: 20118-20132.
    • Related Articles

  • Related Articles

    [1]DAI Yueming, YANG Lufeng, TONG Xiongmin. Real-time Section State Verification Method of Energy Management System Low Voltage Equipment Based on Infrared Image and Deep Learning[J]. Infrared Technology , 2024, 46(12): 1464-1470.
    [2]CHEN Qiuyan, ZHANG Xinyan, HE Min, TIAN Yichun, LIU Ning, GUO Rui, WANG Xiaohui, YOU Siyuan, ZHANG Xiukun. Identification of Pipeline Thermal Image Leakage Based on Deep Learning[J]. Infrared Technology , 2024, 46(5): 522-531.
    [3]DUAN Jin, ZHANG Hao, SONG Jingyuan, LIU Ju. Review of Polarization Image Fusion Based on Deep Learning[J]. Infrared Technology , 2024, 46(2): 119-128.
    [4]FU Tian, DENG Changzheng, HAN Xinyue, GONG Mengqing. Infrared and Visible Image Registration for Power Equipments Based on Deep Learning[J]. Infrared Technology , 2022, 44(9): 936-943.
    [5]ZHANG Yutong, ZHAI Xuping, NIE Hong. Deep Learning Method for Action Recognition Based on Low Resolution Infrared Sensors[J]. Infrared Technology , 2022, 44(3): 286-293.
    [6]ZHONG Rui, YANG Li, DU Yongcheng. The Influence of Deep Transfer Learning Pre-training on Infrared Wake Image Recognition[J]. Infrared Technology , 2021, 43(10): 979-986.
    [7]FAN Peng, FENG Wanxing, ZHOU Ziqiang, ZHAO Chun, ZHOU Sheng, YAO Xiangyu. Application of Deep Learning in Abnormal Insulator Infrared Image Diagnosis[J]. Infrared Technology , 2021, 43(1): 51-55.
    [8]YANG Tao, DAI Jun, WU Zhongjian, JIN Daizhong, ZHOU Guojia. Target Recognition of Infrared Ship Based on Deep Learning[J]. Infrared Technology , 2020, 42(5): 426-433.
    [9]WU Xiao-sheng, SUN Jun-ding. Image Retrieval Based on Improvement Histogram and Spatial Feature[J]. Infrared Technology , 2007, 29(11): 666-669. DOI: 10.3969/j.issn.1001-8891.2007.11.012
    [10]AN Zhi-yong, DU Zhi-Qiang, ZHAO Shan, ZHOU Li-hua. A New Approach for Image Retrieval Based on Color and Spatial Features[J]. Infrared Technology , 2007, 29(6): 361-364. DOI: 10.3969/j.issn.1001-8891.2007.06.013

  • Cited by

    Periodical cited type(15)

    1. 马战南，蒋俊英. 基于LMDI分解法的火电厂热力设备故障自动化诊断技术. 自动化与仪表. 2025(02): 89-92+98 .
    2. 刘传洋，吴一全. 基于红外图像的电力设备识别及发热故障诊断方法研究进展. 中国电机工程学报. 2025(06): 2171-2196 .
    3. 尤渺. 基于模糊Petri网和马尔科夫链理论的水电厂设备故障应急响应模型. 电子设计工程. 2024(10): 59-63 .
    4. 刘沛津，张香瑞，魏平. 基于融合重构的电气设备红外图像EnFCM聚类分割方法. 红外技术. 2024(03): 295-304 . 本站查看
    5. 金萍，侯娟. 面向新型电力系统的粗糙集和双流网络自动化物联设备故障诊断方法研究. 电测与仪表. 2024(09): 166-171 .
    6. 张电 ，张凌跃 ，王宇 ，李伟 ，白困利 . 基于改进Apriori算法的电力物联设备故障安全筛选方法. 自动化与仪器仪表. 2024(09): 190-194 .
    7. 房雪雷，马娟，徐结红，丁津津，彭勃. 基于数字孪生技术的电力设备故障诊断仿真系统构建研究. 自动化与仪器仪表. 2024(10): 233-236 .
    8. 任正国，黄文琦，梁凌宇，郑桦. 基于高低频特征融合的电力设备热故障模糊检测方法. 自动化技术与应用. 2024(11): 39-42+55 .
    9. 陶俊，郭庆，郭力旋，喻成琛. 融合声纹特征的智慧电网主设备故障自动化识别系统. 自动化与仪表. 2024(11): 106-110 .
    10. 杨慢慢，尹兆磊，陈晨，白明辉. 线路多变条件下电力设备热稳定实时控制方法. 自动化与仪表. 2023(08): 26-30 .
    11. 黄志鸿，肖剑，徐先勇，张辉. 基于谱残差变换的电力设备热缺陷识别技术. 红外技术. 2023(08): 884-889 . 本站查看
    12. 刘明哲，毛振攀，周桐，李锦云，乔加奇. 基于LSTM神经网络的发电机绕组散热异常监测系统设计. 环境技术. 2023(07): 69-74 .
    13. 陈凡，金东. 基于大数据的数字化电力设备故障诊断方法. 信息与电脑(理论版). 2023(15): 46-48 .
    14. 杨明祥，李佳宣，殷商莹，贺皎，邹兰青. 基于改进关联规则的电力设备故障预测与诊断. 科学技术创新. 2023(27): 77-80 .
    15. 梁剑，黄志鸿，张可人. 基于多尺度引导滤波和决策融合的电力设备热故障诊断方法研究. 红外技术. 2022(12): 1344-1350 . 本站查看

    Other cited types(2)

Catalog

    Get Citation
    PDF
    XML
    Article views (213) PDF downloads (37) Cited by(17)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return