基于飞行时间法的3D相机研究综述

魏加立; 曲慧东; 王永宪; 朱俊青; 关英俊

基于飞行时间法的3D相机研究综述

1.
长春工业大学机电工程学院，吉林长春 130012
2.
中国科学院长春光学精密机械与物理研究所，吉林长春 130033

基金项目:

国家自然科学基金 11803036

中央引导地方科技发展基金 202002035JC

详细信息

作者简介:
魏加立（1995-），男，硕士研究生，主要从事空间光学遥感器结构优化设计方面研究。E-mail：1569750807@qq.com

通讯作者:
关英俊（1978-），男，博士，教授，博士生导师，主要从事空间光学遥感器结构优化设计方面研究。E-mail：gyj5460@sohu.com

中图分类号: TN219
计量
- 文章访问数: 591
- HTML全文浏览量: 150
- PDF下载量: 138
- 被引次数: 0
出版历程
- 收稿日期: 2020-02-18
- 修回日期: 2020-10-18
- 刊出日期: 2021-01-20

Research Review of 3D Cameras Based on Time-of-Flight Method

1.
School of Mechatronic Engineering, Changchun University of Technology, Changchun 130012, China
2.
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China

摘要

摘要: 基于飞行时间法（Time-of-Flight，TOF）的3D相机是一种体积小、误差小、抗干扰能力强、可直接输出深度信息的新型立体成像设备。目前，该类相机已成为测量成像领域的研究热点。本文首先介绍了TOF相机的发展历程及测量原理；随后对TOF相机测量误差来源及类型进行分析；接着将TOF技术与其他主流的三维成像技术进行对比分析；最后对TOF相机的应用与发展趋势进行了阐述。
- 飞行时间法 /
- 3D-TOF相机 /
- 深度信息 /
- 测量误差
Abstract: The time-of-flight (TOF)-based 3D camera is a new stereo imaging device with a small size, small error, strong anti-interference ability, and direct output of depth information. This type of camera has become a focus of research in the field of measurement and imaging. In this Review, the development history and measurement principle of the TOF camera are introduced. Subsequently, an analysis of the source and type of measurement error of the camera are presented. A comparison of the TOF technology with other mainstream 3D imaging technologies is then provided. Finally, the application and development trends of the TOF camera are described.
- time-of-flight method /
- 3D - TOF camera /
- depth information /
- measurement error

HTML全文

图 1 TOF相机的工作原理

Figure 1. The operational principle of TOF camera

下载: 全尺寸图片幻灯片

图 2 间接测量法相位解调原理图

Figure 2. Schematic diagram of phase demodulation for indirect measurement

下载: 全尺寸图片幻灯片

图 3 深度传感器分类

Figure 3. Depth sensor classification

下载: 全尺寸图片幻灯片

图 4 双目视觉测距示意图

Figure 4. Schematic diagram of binocular visual ranging

下载: 全尺寸图片幻灯片

图 5 结构光基本原理图

Figure 5. Fundamental diagram of structured light

下载: 全尺寸图片幻灯片

图 6 国内外几款主流TOF相机

Figure 6. Several mainstream TOF cameras at home and abroad

下载: 全尺寸图片幻灯片

表 1 三种主流三维成像技术性能对比

Table 1. Performance comparison of three mainstream 3Dimaging technologies

Camera type	Binocular vision	Structured light	TOFcamera
Ranging method	Passive	Active	Active
Precision	medium	Medium -high	medium
Resolution	Medium -high	Medium	low
Frame rate	low	Medium	high
real-time	Medium	low	high
Software complexity	high	Medium	low
Power consumption	low	Medium	Adjustable
Hardware cost	low	high	Medium

下载: 导出CSV

参考文献(43)

[1]	李占利, 周康, 牟琦, 等. TOF相机实时高精度深度误差补偿方法[J/OL].红外与激光工程, 2019(12): 253-262. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201912035.htm LI Zhanli, ZHOU Kang, MOU Qi, et al. Real-time high precision depth error compensation method for TOF camera[J/OL]. Infrared and Laser Engineering, 2019(12): 253-262. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201912035.htm
[2]	Rice K, Moigne J Le, Jain P. Analyzing range maps data for future space robotics applications[C]//Proceedings of the 2nd IEEE International Conference on Space Mission Challenges for Information Technology, 2006, 17: 357-357.
[3]	侯飞.基于飞行时间的三维目标点云提取和重建研究[D].北京: 中国科学院大学(中国科学院国家空间科学中心), 2019. HOU Fei. Research on extraction and reconstruction of three-dimensional target point cloud based on time of flight[D]. Beijing: University of Chinese Academy of Sciences(National Center for space science, Chinese Academy of Sciences), 2019.
[4]	Kohoutek, Tobias. Analysis and processing the 3drangeimagedata for robot monitoring[J]. Geodesy and Cartography, 2008, 34(3): 92-96. doi: 10.3846/1392-1541.2008.34.92-96
[5]	Kuehnle J U, Xue Z, Zoellner J M, et al. Grasping in Depth maps of time-of-flight cameras[C]//International Workshop on Robotic & Sensors Environments. IEEE, 2008: 132-137.
[6]	Oggier T, Lehmann M, Kaufmann R, et al. An all-solid-state optical range camera for 3D real-time imaging with sub-centimeter depth resolution (SwissRanger)[C]//Optical Design and Engineering. International Society for Optics and Photonics, 2004, 5249: 534-545.
[7]	郭宁博, 陈向宁, 薛俊诗.基于飞行时间法的红外相机研究综述[J].兵器装备工程学报, 2017, 38(3): 152-159. doi: 10.11809/scbgxb2017.03.035 GUO Ningbo, CHEN Xiangning, XUE Junshi. Review of research on infrared camera based on time of flight[J]. Journal of Ordnance and Equipment Engineering, 2017, 38(3): 152-159. doi: 10.11809/scbgxb2017.03.035
[8]	丁津津.基于TOF技术的3D相机应用研究综述[C]//中国仪器仪表学会第十二届青年学术会议论文集, 2010: 76-79. DING Jinjin. A review of 3D camera application research based on TOF technology[C]//Proceedings of the 12th Youth Academic Conference of China Instrumentation Society, 2010: 76-79.
[9]	侯飞, 韩丰泽, 李国栋, 等.基于飞行时间的三维成像研究进展和发展趋势[J].导航与控制, 2018, 17(5): 1-7, 48. doi: 10.3969/j.issn.1674-5558.2018.05.001 HOU Fei, HAN Fengze, LI Guodong, et al. Research Progress and Development Trend of 3D Imaging Based on Time of Flight[J]. Navigation and Control, 2018, 17(5): 1-7, 48. doi: 10.3969/j.issn.1674-5558.2018.05.001
[10]	Kaufmann R, Lehmann M, Schweizer M, et al. A time-of-flight line sensor Development and application[J]. Optical Sensing, 2004, 5459: 192-199. doi: 10.1117/12.545571
[11]	Gupta M, Agrawal A, Ashok Veeraraghavan. A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus[J]. International Journal of Computer Vision, 2013, 102(1-3): 33-55. doi: 10.1007/s11263-012-0554-3
[12]	段志坚.基于3D-TOF图像传感器采集系统的设计与实现[D].湘潭: 湘潭大学, 2015. DUAN Zhijian. Design and implementation of 3D-TOF image sensor acquisition system[D]. Xiangtan: Xiangtan University, 2015.
[13]	卢纯青, 宋玉志, 武延鹏, 等.基于TOF计算成像的三维信息获取与误差分析[J].红外与激光工程, 2018, 47(10): 160-166. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201810021.htm LU Chunqing, SONG Yuzhi, WU Yanpeng, et al. 3D Information Acquisition and Error Analysis Based on TOF Computational Imaging[J]. Infrared and Laser Engineering, 2018, 47(10): 160-166. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201810021.htm
[14]	王胤.应用于三维成像飞行时间法建模及其误差分析[D].湘潭: 湘潭大学, 2017. WANG Yin. Application of 3D imaging time-of-flight modeling and its error analysis[D]. Xiangtan: Xiangtan University, 2017.
[15]	杨晶晶, 冯文刚.连续调制TOF图像误差来源及降噪处理[J].合肥工业大学学报:自然科学版, 2012, 35(4): 485-488. https://www.cnki.com.cn/Article/CJFDTOTAL-HEFE201204013.htm YANG Jingjing, FENG Wengang. The source of continuous modulation TOF image error and noise reduction processing[J]. Journal of Hefei University of Technology: Natural Science, 2012, 35(4): 485-488. https://www.cnki.com.cn/Article/CJFDTOTAL-HEFE201204013.htm
[16]	Payne A D, Dorrington A A, Cree M J. Illumination waveform optimization for time-of-flight range imaging cameras[C]//Videometrics, Range Imaging, and Applications XI. International Society for Optics and Photonics, 2011, 8085: 136-148.
[17]	Corti A, Giancola S, Mainetti G, et al. A metrological characterization of the Kinect V2 time-of-flight camera[J]. Robotics and Autonomous Systems, 2016, 75(PB): 584-594.
[18]	Rapp H. Faculty for physics and astronomy[D]. Heidelberg: University of Heidelberg, 2007.
[19]	Ruocco R, White T, Jarrett D. Systematic errors in active 3D vision sensors[J]. IEEE Proceedings-Vision, Image and Signal Processing, 2003, 150(6): 341-345. doi: 10.1049/ip-vis:20030794
[20]	Jung J, Lee J, Jeong Y, et al. Time-of-Flight Sensor Calibration for a Color and Depth Camera Pair[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(7): 1501-1513. doi: 10.1109/TPAMI.2014.2363827
[21]	Fuchs S, Hirzinger G. Extrinsic and depth calibration of ToF-cameras[C]//IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, 2008: 1-6.
[22]	潘华东, 谢斌, 刘济林.无扫描激光三维成像系统的建模与仿真[J].浙江大学学报:工学版, 2010(4): 21-26, 33. https://www.cnki.com.cn/Article/CJFDTOTAL-ZDZC201004002.htm PAN Huadong, XIE Bin, LIU Jilin. Modeling and Simulation of Scanless Laser 3D Imaging System[J]. Journal of Zhejiang University: Engineering Science, 2010(4): 21-26, 33. https://www.cnki.com.cn/Article/CJFDTOTAL-ZDZC201004002.htm
[23]	梁斌, 何英, 邹瑜, 等. ToF相机在空间非合作目标近距离测量中的应用[J].宇航学报, 2016, 37(9): 1080-1088. doi: 10.3873/j.issn.1000-1328.2016.09.007 LIANG Bin, HE Ying, ZOU Yu, et al. Application of ToF camera in close measurement of non-cooperative objects in space[J]. Acta Astronautica, 2016, 37(9): 1080-1088. doi: 10.3873/j.issn.1000-1328.2016.09.007
[24]	Opower H. Multiple view geometry in computer vision[J]. Optics and Lasers in Engineering, 2002, 37(1): 85-86. doi: 10.1016/S0143-8166(01)00145-2
[25]	张鑫, 习俊通.基于随机编码结构光的双目立体三维测量系统[J].机电一体化, 2013, 19(3): 61-65. doi: 10.3969/j.issn.1007-080x.2013.03.013 ZHANG Xin, XI Juntong. Binocular three-dimensional measurement system based on randomly encoded structured light[J]. Mechatronics, 2013, 19(3): 61-65. doi: 10.3969/j.issn.1007-080x.2013.03.013
[26]	Moreno-Noguer F, Belhumeur P N, Nayar S K. Active refocusing of images and videos[J]. ACM Transactions on Graphics, 2007, 26(99): 67. doi: 10.1145/1276377.1276461
[27]	Prusak A, Melnychuk O, Roth H, et al. Pose estimation and map building with a Time-Of-Flight camera for robot navigation[J]. International Journal of Intelligent Systems Technologies and Applications, 2008, 5(3/4): 355-364. doi: 10.1504/IJISTA.2008.021298
[28]	Alenyà G, Foix S, Torras C. Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments[J]. Intelligent Service Robotics, 2014, 7(4): 211-220. doi: 10.1007/s11370-014-0159-5
[29]	司秀娟.深度图像处理在精准农业领域的应用研究[D].合肥: 中国科学技术大学, 2017. SI Xiujuan. Application Research of Deep Image Processing in Precision Agriculture[D]. Hefei: University of Science and Technology of China, 2017.
[30]	Vázquez-Arellano, Manuel Reiser D, Paraforos D S, et al. 3-D reconstruction of maize plants using a time-of-flight camera[J]. Computers and Electronics in Agriculture, 2018, 145: 235-247 doi: 10.1016/j.compag.2018.01.002
[31]	Penne J, Schaller C, Hornegger J, et al. Robust real-time 3D respiratory motion detection using time-of-flight cameras[J]. International Journal of Computer Assisted Radiology and Surgery, 2008, 3(5): 427-431. doi: 10.1007/s11548-008-0245-2
[32]	Soutschek S, Penne J, Hornegger J, et al. 3-D gesture-based scene navigation in medical imaging applications using Time-of-Flight cameras[C]//IEEE Conf. on Computer Vision & Pattern Recogn., 2008: 1-6.
[33]	雷禧生, 肖昌炎, 蒋仕龙.基于TOF相机的喷涂工件在线三维重建[J].电子测量与仪器学报, 2017, 31(12): 1991-1998. https://www.cnki.com.cn/Article/CJFDTOTAL-DZIY201712018.htm LEI Xisheng, XIAO Changyan, JIANG Shilong. Online 3d reconstruction of sprayed workpiece based on TOF camera[J]. Journal of Electronic Measurement and Instrument, 2017, 31(12): 1991-1998. https://www.cnki.com.cn/Article/CJFDTOTAL-DZIY201712018.htm
[34]	Ahmad R, Plapper P. Generation of safe tool-path for 2.5D milling/drilling machine-tool using 3D ToF sensor[J]. CIRP Journal of Manufacturing Science and Technology, 2015, 10: 84-91. doi: 10.1016/j.cirpj.2015.04.003
[35]	丁津津. TOF三维摄像机的误差分析及补偿方法研究[D].合肥: 合肥工业大学, 2011. DING Jinjin. Error analysis and compensation method of TOF 3d camera[D]. Hefei: Hefei University of Technology, 2011.
[36]	王亚洲.采用TOF面阵传感器与双目视觉融合的三维深度相机设计[D].深圳: 深圳大学, 2017. WANG Yazhou. Design of a 3D Depth Camera Using TOF Area Array Sensor and Binocular Vision[D]. Shenzhen: Shenzhen University, 2017.
[37]	黄舒兰. ToF与立体视觉技术相结合的三维重建方法研究[D].深圳: 中国科学院大学(中国科学院深圳先进技术研究院), 2019. HUANG Shulan. Research on 3D reconstruction method combining ToF and stereo vision technology[D]. Shenzhen: University of Chinese Academy of Sciences (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), 2019.
[38]	Hullin M B. Computational imaging of light in flight[C]//Proceedings of Optoelectronic Imaging and Multimedia Technology Ⅲ, 2014: 927314.
[39]	XIE Jun, Feris Rogerio Schmidt, SUN Mingting. Edge-Guided Single Depth Image Super Resolution[J]. IEEE Transactions on Image Processing: a publication of the IEEE Signal Processing Society, 2016, 25(1): 428-438. doi: 10.1109/TIP.2015.2501749
[40]	SONG X B, DAI Y C, QIN X Y. Deep Depth Super-Resolution: Learning Depth Super-Resolution using Deep Convolutional Neural Network[C]//Asian Conference on Computer Vision, 2017, 10114: 360-376.
[41]	Kahlmann T, Oggier T, Lustenberger F, et al. 3D-TOF sensors in the automobile[C]//Proceedings of SPIE the International Society for Optical Engineering, 2005, 5663: 216-224.
[42]	宋玉志, 卢纯青, 王立. 3D-TOF相机在空间近距离目标探测中的应用研究[J].空间控制技术与应用, 2019, 45(1): 53-59. https://www.cnki.com.cn/Article/CJFDTOTAL-KJKZ201901021.htm SONG Yuzhi, LU Chunqing, WANG Li. Research on the application of cameras in the detection of close range objects in space[J]. Aerospace Control and Application, 2019, 45(1): 53-59. https://www.cnki.com.cn/Article/CJFDTOTAL-KJKZ201901021.htm
[43]	Nguyen Trong-Nguyen, Huynh Huu-Hung, Meunier Jean. Human gait symmetry assessment using a depth camera and mirrors[J]. Computers in Biology and Medicine, 2018, 101: 174-183. doi: 10.1016/j.compbiomed.2018.08.021