视觉SLAM在动态场景下的图像处理方法

游通飞; 孔令华; 刘文玉; 易定容; 殷江

视觉SLAM在动态场景下的图像处理方法

游通飞^{1, 2},
孔令华^{1, 2, ,},
刘文玉¹,
易定容³,
殷江^{1, 2}

1.
福建工程学院机械与汽车工程学院，福建福州 350118
2.
福建工程学院数字福建工业制造物联网实验室，福建福州 350118
3.
华侨大学机电及自动化学院，福建厦门 361021

基金项目:

国家自然科学基金资助项目 51775200

详细信息

作者简介:
游通飞（1994-），男，福建福州人，硕士研究生，研究方向：视觉SLAM和语义分割

通讯作者:
孔令华（1963-），男，加拿大人，教授，博士，硕士生导师，研究方向：三维视觉和多光谱检测。E-mail：15392030898@163.com

中图分类号: TP391
计量
- 文章访问数: 173
- HTML全文浏览量: 36
- PDF下载量: 37
- 被引次数: 0
出版历程
- 收稿日期: 2020-06-06
- 修回日期: 2020-07-28
- 刊出日期: 2021-10-20

Image Processing Method for Visual Simultaneous Localization and Mapping

YOU Tongfei^{1, 2},
KONG Linghua^{1, 2
, ,},
LIU Wenyu¹,
YI Dingrong³,
YIN Jiang^{1, 2}

1.
School of Mechanical and Automotive Engineering, Fujian University of Technology, Fuzhou 350118, China
2.
Digital Fujian Industrial Manufacturing IoT Lab, Fuzhou 350118, China
3.
College of Mechanical Engineering and Automation, Huaqiao University, Xiamen 361021, China

摘要

摘要: SLAM一直是机器人领域的研究热点，近年来取得了万众瞩目的进步，但很少有SLAM算法考虑到动态场景的处理。针对视觉SLAM场景中动态目标的处理，提出一种在动态场景下的图像处理方法。将基于深度学习的语义分割算法引入到ORB_SLAM2方法中，对输入图像进行分类处理的同时剔除人身上的特征点。基于已经剔除特征点的图像进行位姿估计。在TUM数据集上与ORB_SLAM2进行对比，在动态场景下的绝对轨迹误差和相对路径误差精度提高了90%以上。在保证地图精度的前提下，改善了地图的适用性。
- 视觉SLAM /
- 动态场景 /
- ORB-SLAM2 /
- 特征点 /
- 剔除
Abstract: Simultaneous localization and mapping(SLAM) has always been a research hotspot in the robotics field. In recent years, remarkable progress has been made in SLAM research, but few SLAM algorithms have considered the processing of dynamic scenes. Therefore, in this study, an image processing method for dynamic target processing in a visual SLAM scene is proposed. The semantic segmentation algorithm based on deep learning was introduced into the ORB_SLAM2 method and input image classification processing was accomplished while removing the feature points on the body. Pose estimation was performed based on images with eliminated feature points. Compared to ORB_SLAM2 on the TUM dataset, the absolute trajectory error and relative path error accuracy were improved by more than 90% in the dynamic scene. To ensure the accuracy of the generated map, the applicability of the map was improved.
- visual SLAM /
- dynamic scene /
- ORB-LAM2 /
- feature points /
- remove

HTML全文

图 1 ORB_SLAM2系统线程和结构

Figure 1. ORB_SLAM2 system threads and structure

下载: 全尺寸图片幻灯片

图 2 PSPNet算法结构

Figure 2. PSPNet algorithm structure

下载: 全尺寸图片幻灯片

图 3 剔除动态点方法

Figure 3. Remove dynamic point method

下载: 全尺寸图片幻灯片

图 4 ORB-SLAM2在walking_xyz下的误差（左：轨迹误差；右：相对位姿误差）

Figure 4. Error of ORB-SLAM2 under walking_xyz(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 5 本文方法在walking_xyz下的误差（左：轨迹误差；右：相对位姿误差）

Figure 5. Error of our method under walking_xyz(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 6 ORB-SLAM2在walking_halfsphere下的误差（左：轨迹误差；右：相对位姿误差）

Figure 6. Error of ORB-SLAM2 under walking_halfsphere(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 7 本文方法在walking_halfsphere下的误差（左：轨迹误差；右：相对位姿误差）

Figure 7. Error of our method under walking_halfsphere(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 8 ORB-SLAM2在walking_static下的误差（左：轨迹误差；右：相对位姿误差）

Figure 8. Error of ORB-SLAM2 under walking_static(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 9 本文方法在walking_static下的误差（左：轨迹误差；右：相对位姿误差）

Figure 9. Error of our method under walking_static(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 10 ORB-SLAM2在sitting_static下的误差（左：轨迹误差；右：相对位姿误差）

Figure 10. Error of ORB-SLAM2 under sitting_static(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

图 11 本文方法在sitting_static下的误差（左：轨迹误差；右：相对位姿误差）

Figure 11. Error of our method under sitting_static(left: absolute trajectory error; right: relative pose error)

下载: 全尺寸图片幻灯片

表 1 绝对轨迹误差对比（ATE）

Table 1. Absolute trajectory error comparison

Sequences	ORB-SLAM2				Ours				Improvement/%
Sequences	Rmse	Mean	Median	Std	Rmse	Mean	Median	Std	Rmse	Mean	Median	Std
walking_xyz	0.5357	0.4964	0.4733	0.2014	0.0269	0.0185	0.0151	0.0196	94.98	96.27	96.81	90.27
walking_halfsphere	0.4318	0.3651	0.3107	0.2305	0.0334	0.0285	0.0243	0.0175	92.26	92.19	92.18	92.41
walking_static	0.3753	0.3398	0.2963	0.1593	0.0076	0.0068	0.0062	0.0034	97.97	98.00	97.91	97.87
sitting_static	0.0082	0.0071	0.0063	0.0041	0.0062	0.0054	0.0047	0.0031	24.39	23.94	25.40	24.39

下载: 导出CSV

表 2 相对位姿误差对比（RPE）

Table 2. Relative pose error comparison

Sequences	ORB-SLAM2				Ours				Improvement/%
Sequences	Rmse	Mean	Median	Std	Rmse	Mean	Median	Std	Rmse	Mean	Median	Std
walking_xyz	0.7856	0.6444	0.5714	0.4493	0.0400	0.0280	0.0222	0.0285	94.91	95.65	96.11	93.66
walking_halfsphere	0.6200	0.4957	0.4705	0.3724	0.0474	0.0414	0.0373	0.0231	92.35	91.65	92.07	93.80
walking_static	0.5354	0.3946	0.1802	0.3618	0.0112	0.0102	0.0096	0.0048	97.91	97.42	94.67	98.67
sitting_static	0.0127	0.0112	0.0101	0.0060	0.0093	0.0081	0.0073	0.0044	26.77	27.68	27.72	26.67

下载: 导出CSV

表 3 相对旋转误差对比（RRE）

Table 3. Relative rotation error comparison

Sequences	ORB-SLAM2				Ours				Improvement/%
Sequences	Rmse	Mean	Median	Std	Rmse	Mean	Median	Std	Rmse	Mean	Median	Std
walking_xyz	14.3695	11.7969	0.1856	8.2046	0.8776	0.6192	0.0082	0.6220	93.89	94.75	95.58	92.42
walking_halfsphere	14.5176	12.0261	0.2177	8.1323	1.0316	0.8956	0.0139	0.5118	92.89	92.55	93.62	93.71
walking_static	9.6864	7.1088	0.0558	6.5796	0.3021	0.2724	0.0044	0.1306	96.88	96.17	92.11	98.02
sitting_static	0.3572	0.3220	0.0054	0.1546	0.3347	0.2981	0.0048	0.1523	6.30	7.42	11.11	1.49

下载: 导出CSV

表 4 两种方法在TUM数据集的耗时

Table 4. Time consuming of the two methods in TUM dataset

Methods	1	2	3	Average
ORB-SLAM2	54.314	58.629	59.373	57.439
Ours	81.241	79.298	78.505	79.681

下载: 导出CSV

参考文献(13)

[1]	Durrant-Whyte H, Bailey Tim. Simultaneous localization and mapping: part I[J]. IEEE Robotics & Amp Amp Automation Magazine, 2006, 13(2): 99-10. http://vigir.missouri.edu/~gdesouza/Research/MobileRobotics/Simultaneous%20localization%20and%20mapping.%20IEEE%20Robotics%20and%20Automation%20magazine-%20Part%201,%20H.%20Durrant-White%20and%20T.%20Bailey.pdf
[2]	张金凤, 石朝侠, 王燕清. 动态场景下基于视觉特征的SLAM方法[J]. 计算机工程, 2020, 46(10): 95-102. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC202010012.htm ZHANG Jinfeng, SHI Chaoxia, WANG Yanqing. SLAM method based on visual features in dynamic scene[J]. Computer Engineering, 2020, 46(10): 95-102. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC202010012.htm
[3]	江国来. 共融移动服务机器人导航与交互关键技术研究[D]. 深圳: 中国科学院大学(中国科学院深圳先进技术研究院), 2019. JIANG Guolai. Research on Key Technologies of Navigation and Interaction of Inclusive Mobile Service Robot[D]. Shenzhen: University of Chinese Academy of Sciences, 2019.
[4]	HAN Shuangquan, XI Zhihong. Dynamic scene semantics SLAM based on semantic segmentation[J]. IEEE Access, 2020, 8: 43563-43570. doi: 10.1109/ACCESS.2020.2977684
[5]	WANG Kai, LIN Yimin, WANG Luowei, et al. A unified framework for mutual improvement of SLAM and semantic segmen- tation[C]//International Conference on Robotics and Automation (ICRA) of IEEE, 2019: 5224-5230.
[6]	Bescos Berta, Fácil José M, Civera Javier, et al. Dyna SLAM: tracking, mapping and in painting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083. doi: 10.1109/LRA.2018.2860039
[7]	YU Chao, LIU Zuxin, LIU Xinjun, et al. DS-SLAM: a semantic visual SLAM towards dynamic environments[C]//IEEE International Conference on Intelligent Robots and Systems(IROS), 2018: 1168-1174.
[8]	王召东, 郭晨. 一种动态场景下语义分割优化的ORB_SLAM2[J]. 大连海事大学学报, 2018, 44(4): 121-126. doi: 10.3969/j.issn.1671-7031.2018.04.020 WANG Zhaodong, GUO Chen. An improved ORB_SLAM2 in dynamic scene with semantic segmentation[J]. Journal of Dalian Maritime University, 2018, 44(4): 121-126. doi: 10.3969/j.issn.1671-7031.2018.04.020
[9]	王晨, 汤心溢, 高思莉. 基于深度卷积神经网络的红外场景理解算法[J]. 红外技术, 2017, 39(8): 728-733. http://hwjs.nvir.cn/article/id/hwjs201708010 WANG Chen, TANG Xinyi, GAO Sili. Infrared scene understanding algorithm based on deep convolutional neural network[J]. Infrared Technology, 2017, 39(8): 728-733. http://hwjs.nvir.cn/article/id/hwjs201708010
[10]	Mur-Artal Raul, D Tardos Juan. ORB-SLAM2: an opensource slam system for monocular, stereo, and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262. doi: 10.1109/TRO.2017.2705103
[11]	ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6230-6239.
[12]	李瀚超, 蔡毅, 王岭雪. 全局特征提取的全卷积网络图像语义分割算法[J]. 红外技术, 2019, 41(7): 595-599, 615. http://hwjs.nvir.cn/article/id/hwjs201907001 LI Hanchao, CAI Yi, WANG Lingxue. Image semantic segmentation based on fully convoluted network with global feature extraction[J]. Infrared Technology, 2019, 41(7): 595-599. http://hwjs.nvir.cn/article/id/hwjs201907001
[13]	Sturm Jürgen, Engelhard Nikolas, Endres Felix, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//RSJ Int. Conf. Intell. Robots Syst. of IEEE, 2012: 573-580.