[1]李想,孙韶媛,刘训华,等.基于ConvLSTM双通道编码网络的夜间无人车场景预测[J].红外技术,2020,42(8):789-794.[doi:10.11846/j.issn.1001_8891.202008014]
 LI Xiang,SUN Shaoyuan,LIU Xunhua,et al.Dual-Channel Encoding Network Based on ConvLSTM for Driverless Vehicle Night Scene Prediction[J].Infrared Technology,2020,42(8):789-794.[doi:10.11846/j.issn.1001_8891.202008014]
点击复制

基于ConvLSTM双通道编码网络的夜间无人车场景预测
分享到:

《红外技术》[ISSN:1001-8891/CN:CN 53-1053/TN]

卷:
42卷
期数:
2020年第8期
页码:
789-794
栏目:
出版日期:
2020-08-23

文章信息/Info

Title:
Dual-Channel Encoding Network Based on ConvLSTM for Driverless Vehicle Night Scene Prediction

文章编号:
1001-8891(2020)08-0789-06
作者:
李想12孙韶媛12刘训华12顾立鹏12
1. 东华大学 信息科学与技术学院;
2. 东华大学 数字化纺织服装技术教育部工程研究中心

Author(s):
LI Xiang12SUN Shaoyuan12LIU Xunhua12GU Lipeng12
1. College of Information Science and Technology, Donghua University;
2. Engineering Research Center of Digitized Textile & Fashion Technology, Ministry of Education, Donghua University

关键词:
红外图像场景预测卷积长短时记忆编码网络
Keywords:
infrared image scene prediction convolutional long-short term memory encoder network
分类号:
TN219
DOI:
10.11846/j.issn.1001_8891.202008014
文献标志码:
A
摘要:
为了提高夜间无人车驾驶的决策速度,减少夜间交通事故发生的概率,对无人驾驶场景预测任务进行了研究。提出了基于卷积长短时记忆的双通道编码夜间无人车场景预测网络,利用两个子网络:时间子网络提取红外视频序列的时序特征,空间子网络提取红外图像的空间特征,通过融合网络融合特征,输入到解码网络中,以实现对红外视频的未来帧预测。该网络具有端到端的优点,能够实现输入视频序列,直接输出预测帧的图像,并可以预测多帧图像。实验结果表明,该网络对夜间场景预测较准确,可以预测未来1.2 s后的图像,预测速度快,为0.02 s/帧,达到了实时性要求。
Abstract:
The task of scene prediction is studied to improve the decision-making speed of driverless vehicles for reducing the probability of traffic accidents at night. A dual-channel encoding night scene prediction network is proposed based on a convolutional long-short term memory network. First, the temporal features of infrared video sequences and the spatial features of infrared images are extracted by the temporal and spatial sub-networks, respectively. Second, spatial-temporal features obtained by the fusion network are input into the decoding network to predict future frames of infrared video. This is an end-to-end network and can predict multiple frames. The experimental results show that the proposed network is more accurate in night scene prediction and can predict images 1.2 s in the future with a fast prediction speed of 0.02 s/frame, which fulfills the real-time requirement.

参考文献/References:

[1]? YUSUF Aytar, CARL Vondrick, ANTONIO Torralba. Learning sound representations from unlabeled video[C]//Conference and Workshop on Neural Information Processing Systems, 2016: 892-900.
[2]? 莫凌飞, 蒋红亮, 李煊鹏. 基于深度学习的视频预测研究综述[J]. 智能系统学报, 2018, 13(1): 85-96.
MO Lingfei, JIANG Hongliang, LI Xuanpeng. Review of video prediction research based on deep learning[J]. CAAI Transactions on Intelligent Systems, 2018, 13(1): 85-96.
[3]? 潘福全, 亓荣杰, 张璇, 等. 无人驾驶汽车研究综述与发展展望[J]. 科技创新与应用, 2017(2): 27-28.
PAN Fuquan, QI Rongjie, ZHANG Xuan, et al. Research and development prospects of driverless vehicles[J]. Technological Innovation and Application, 2017(2): 27-28.
[4]? LI Qing, HE Tao, FU Guodong. Judgment and optimization of video image recognition in obstacle detection in intelligent vehicle[J]. Mechanical Systems and Signal Processing, 2020, 136: 106406
[5]? 石永彪, 张湧. 车载红外夜视技术发展研究综述[J]. 红外技术, 2019, 41(6): 504-510.
SHI Yongbiao, ZHANG Yong. Review of research on development of vehicle infrared night vision technology[J]. Infrared Technology, 2019, 41(6): 504-510.
[6]? 易诗, 聂焱, 张洋溢, 等. 基于红外热成像与YOLOv3的夜间目标识别方法[J]. 红外技术, 2019, 41(10): 970-975.
YI Shi, NIE Yan, ZHANG Yangyi, et al. Night target recognition method based on infrared thermal imaging and YOLOv3[J]. Infrared Technology, 2019, 41(10): 970-975.
[7]? WANG T C, LIU M Y, ZHU J Y, et al. Video-to-video synthesis[J]//Conference and Workshop on Neural Information Processing Systems, 2018: 660-613.
[8]? PAN Junting, WANG Chengyu, JIA Xu, et al. Video generation from single semantic label map[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2019: 3728-3737.
[9]? WILLIAM Lotter, GABRIEL Kreiman, DAVID Cox. Deep predicitve coding networks for video prediction and unsupervised learning[C]// International Conference on Learning Representations(ICLR), 2017: 78-95.
[10]? WANG F Y, ZHENG N N, CAO D, et al. Parallel driving in CPSS: a unified approach for transport automation and vehicle intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2017, 4(4): 577-587.
[11]? LAWRENCE S , GILES C L , TSOI A C, et al. Face recognition: a convolutional neural-network approach[J]. IEEE Transactions on Neural Networks, 1997, 8(1): 98-113.
[12]? HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[13]? HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016: 770-778 .
[14]? ZHOU Wang, ALAN Bovik, HAMID Sheikh, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 14(4): 600-612.
[15]? SHI Xingjian, CHEN Zhourong, WANG Hao, et al. Convolutional LSTM network: a machine learning approach for precipitation now casting[C]//Conference and Workshop on Neural Information Processing Systems, 2015: doi: 10.1007/978-3-319-21233-3_6.
[16]? ALEX Lee, RICHARD Zhang, FREDERIK Ebert, et al. Stochastic adversarial video prediction[DB/OL]. 2018: arXiv:1804.01523https:// arxiv.org/abs/1804.01523v1.

相似文献/References:

[1]郭水旺,王宝红,季钢,等.基于基因表达式编码算法的红外图像轮廓提取[J].红外技术,2013,35(01):038.
 GUO Shui-wang,WANG Bao-hong,JI Gang,et al. Infrared Image Contour Extraction Based on the Gene Expression Coding Algorithm[J].Infrared Technology,2013,35(8):038.
[2]孙爱平,皮冬明,安长亮,等. 光机装校阶段红外与可见光图像配准技术研究[J].红外技术,2013,35(01):050.
 SUN Ai-ping,PI Dong-ming,AN Chang-liang,et al. Study on IR/Visible Image Registration for Lens Assembly[J].Infrared Technology,2013,35(8):050.
[3]路建方,王新赛,肖志洋,等. 基于FPGA的红外图像自适应分段线性增强算法[J].红外技术,2013,35(02):102.
 LU Jian-fang,WANG Xin-sai,XIAO Zhi-yang,et al. An Adaptive Piecewise Linear Enhance Algorithm for Infrared Image Based on FPGA[J].Infrared Technology,2013,35(8):102.
[4]徐铭蔚,李郁峰,陈念年,等.多尺度融合与非线性颜色传递的微光与红外图像染色[J].红外技术,2012,34(12):722.
 XU Ming-wei,LI Yu-feng,CHEN Nian-nian,et al. Coloration of the Low Light Level and Infrared Image Using Multi-scale Fusion and Nonlinear Color Transfer Technique[J].Infrared Technology,2012,34(8):722.
[5]纪利娥,杨风暴,王志社,等. 基于边缘图像和SURF特征的可见光与红外图像的匹配算法[J].红外技术,2012,34(11):629.
 JI Li-e,YANG Feng-bao,WANG Zhi-she,et al.Visible and Infrared Image Matching Algorithm Based on Edge Image and SURF Features[J].Infrared Technology,2012,34(8):629.
[6]张红辉,罗海波,余新荣,等. 改进的神经网络红外图像非均匀性校正方法[J].红外技术,2013,35(04):232.
[7]张强,侯宁,刘红燕. 红外焦平面阵列非均匀性多点实时压缩校正研究[J].红外技术,2012,34(10):593.
 ZHANG Qiang,HOU Ning,LIU Hong-yan. Study on Real-time Multi-points Compressive Nonuniformity Correction of IRFPA[J].Infrared Technology,2012,34(8):593.
[8]路建方,王新赛,肖志洋,等. 基于灰度分层的FPGA红外图像伪彩色实时化研究[J].红外技术,2013,35(05):285.
 LU Jian-fang,WANG Xin-sai,XIAO Zhi-yang,et al. The Research on Real-time Pseudo-color of Infrared Image in FPGA Based on Gray Delaminating[J].Infrared Technology,2013,35(8):285.
[9]陈钱.红外图像处理技术现状及发展趋势[J].红外技术,2013,35(06):311.
 CHEN Qian.The Status and Development Trend of Infrared Image Processing Technology[J].Infrared Technology,2013,35(8):311.
[10]谭东杰,张安.基于局部直方图规定化的红外图像非均匀性校正[J].红外技术,2013,35(06):325.
 TAN Dong-jie,ZHANG An.Non-uniformity Correction Based on Local Histogram Specification[J].Infrared Technology,2013,35(8):325.

备注/Memo

备注/Memo:
收稿日期:2019-12-28;修订日期:2020-02-13.
作者简介:李想(1995-),女,山东临沂人,硕士,主要研究方向为红外图像处理与深度学习。E-mail:xiangxlily@163.com。
基金项目:上海市科委基础研究项目(15JC1400600)。

更新日期/Last Update: 2020-08-20