留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

稀疏激光雷达与可见光/红外成像系统的标定方法

刘宇 蔡毅 戎宁涛 周云扬 王岭雪

刘宇, 蔡毅, 戎宁涛, 周云扬, 王岭雪. 稀疏激光雷达与可见光/红外成像系统的标定方法[J]. 红外技术, 2022, 44(12): 1264-1272.
引用本文: 刘宇, 蔡毅, 戎宁涛, 周云扬, 王岭雪. 稀疏激光雷达与可见光/红外成像系统的标定方法[J]. 红外技术, 2022, 44(12): 1264-1272.
LIU Yu, CAI Yi, RONG Ningtao, ZHOU Yunyang, WANG Lingxue. Calibration Between Sparse LIDAR and Visible/Infrared Imaging Systems[J]. Infrared Technology , 2022, 44(12): 1264-1272.
Citation: LIU Yu, CAI Yi, RONG Ningtao, ZHOU Yunyang, WANG Lingxue. Calibration Between Sparse LIDAR and Visible/Infrared Imaging Systems[J]. Infrared Technology , 2022, 44(12): 1264-1272.

稀疏激光雷达与可见光/红外成像系统的标定方法

详细信息
    作者简介:

    刘宇(1997-),男, 辽宁沈阳人,硕士研究生,主要从事激光雷达与图像处理的研究。E-mail: andyliu0881@163.com

    通讯作者:

    王岭雪(1973-),女,云南石屏人,副教授,博士,主要从事红外成像、图像处理和红外光谱等方面的研究。E-mail:neobull@bit.edu.cn

  • 中图分类号: TP249

Calibration Between Sparse LIDAR and Visible/Infrared Imaging Systems

  • 摘要: 激光雷达与成像系统之间的位姿标定是激光点云与图像像素进行融合的前提。目前主流的离线标定方法中,普通棋盘格标定板用于64线及以上的激光雷达时效果较好,而用于16线激光雷达时由于其数据稀疏而导致误差较大。而且,涉及红外成像系统的标定时,需要特制的棋盘格来获得发射率差异。本文针对稀疏激光雷达点云数据较少的问题,研究了可以同时标定激光雷达与可见光、红外成像系统的方法,设计了菱形九孔标定板,并提出几何约束损失函数来优化特征点的坐标。最后,分别使用红外和可见光成像系统与16线激光雷达进行标定,实验结果表明,平均重投影误差均在3个像素之内,取得了较好的效果。本文方法还能用于稀疏激光雷达与可见光-红外多波段成像系统的标定。
  • 图  1  本文设计的标定板

    Figure  1.  The calibration board designed by this paper

    图  2  激光雷达数据计算圆心坐标

    Figure  2.  Calculating circle center coordinates using LIDAR data

    图  3  标定实验装置

    Figure  3.  Calibration experimental equipment

    图  4  实验场景及数据

    Figure  4.  Experimental scene and data

    图  5  重投影误差

    Figure  5.  Reprojection error

    图  6  激光雷达点云投影结果

    Figure  6.  Results of LIDAR point cloud projection

    表  1  两种方法初值误差对比

    Table  1.   Comparison of initial value errors of two methods

    3D Constraint Method (a) PnP Method (b)
    Number Rotary axis error(a) Angular error (a) Translation error (a) Rotary axis error(b) Angular error (b) Translation error (b)
    1 6.0×10−5 5×10−5 2.3×10−5 0.890 0.165 0.376
    2 6.4×10−6 8×10−5 3.1×10−5 0.593 0.067 0.434
    3 2.9×10−5 8×10−5 5.3×10−5 0.283 0.038 0.387
    4 6.1×10−5 3×10−5 4.9×10−5 0.733 0.048 0.218
    5 5.7×10−5 8×10−5 7.7×10−5 0.710 0.070 0.520
    6 3.4×10−5 1×10−4 7.2×10−5 0.904 0.022 0.615
    7 5.9×10−5 7×10−5 5.5×10−5 0.278 0.036 0.497
    8 3.9×10−5 3×10−5 3.9×10−5 0.820 0.021 0.581
    9 5.3×10−5 9×10−6 2.9×10−5 0.718 0.027 0.480
    Average 4.4×10−5 6×10−6 4.7×10−5 0.658 0.054 0.456
    下载: 导出CSV

    表  2  本文方法的仿真结果

    Table  2.   The simulation results of our method

    Number Lidar loss
    before optimization
    Lidar loss
    after optimization
    Camera loss
    before optimization
    Camera loss
    after optimization
    Rotary axis error Angular error Translation error
    1 4.9377 0.0147 4.3755 0.0010 5.9×10−3 0.0025 3.6×10−3
    2 4.6482 0.0333 4.6580 0.0030 4.2×10−3 0.0001 6.9×10−3
    3 4.1757 0.0054 4.2307 0.0052 5.310−3 0.0003 5.8×10−3
    4 4.4703 0.0009 4.6283 0.0034 4.3×10−3 0.0046 8.9×10−3
    5 4.7989 0.0010 4.4958 0.0011 7.1×10−3 0.0082 79×10−3
    6 4.8751 0.0054 4.1330 0.0018 7.1×10−3 0.0162 6.9×10−3
    7 4.2961 0.0035 4.4130 0.0095 8.4×10−3 0.0224 5.1×10−3
    8 4.4903 0.0013 4.4346 0.0073 8.9×10−3 0.0043 1.1×10−2
    9 4.4642 0.0018 4.5932 0.0153 7.3×10−3 0.0142 9.9×10−3
    Average 4.5729 0.0075 4.4402 0.0053 6.5×10−3 0.0081 7.3×10−3
    下载: 导出CSV

    表  3  可见光系统实验结果

    Table  3.   Experimental results of visible imaging system

    No. Rotation matrix Rlc (c) Translation vector tlc(c) Reprojection error (c)/pixel Rotation matrix Rlc(d) Translation vector tlc(d) Reprojection error (d)/pixel
    1 $ \left[ {\begin{array}{*{20}{c}} {0.9905}&{ - 0.1364}&{ - 0.0192} \\ { - 0.0138}&{0.0403}&{ - 0.9991} \\ {0.1371}&{0.9898}&{0.0381} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0172} \\ {0.0660} \\ {0.0116} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:4.7595} \\ {y:2.7523} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.9882}&{ - 0.1524}&{ - 0.0156} \\ { - 0.0132}&{0.0166}&{ - 0.9998} \\ {0.1526}&{0.9882}&{0.0144} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0172} \\ {0.0660} \\ {0.0116} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.3598} \\ {y:2.0767} \end{array} $
    2 $ \left[ {\begin{array}{*{20}{c}} {0.9906}&{ - 0.1358}&{ - 0.0111} \\ { - 0.0104}&{0.0063}&{ - 0.9999} \\ {0.1359}&{0.9907}&{0.0049} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0165} \\ {0.0188} \\ {0.0200} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.5315} \\ {y:5.4321} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.9859}&{ - 0.1661}&{ - 0.0205} \\ { - 0.0149}&{0.0350}&{ - 0.9993} \\ {0.1667}&{0.9855}&{0.0320} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0165} \\ {0.0188} \\ {0.0200} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.6225} \\ {y:2.0955} \end{array} $
    3 $ \left[ {\begin{array}{*{20}{c}} {0.9924}&{ - 0.1224}&{ - 0.0114} \\ { - 0.0100}&{0.0119}&{ - 0.9998} \\ {0.1225}&{0.9924}&{0.0106} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0233} \\ {0.0664} \\ {0.0262} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.2698} \\ {y:4.5639} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.9883}&{ - 0.1513}&{ - 0.0204} \\ { - 0.0179}&{0.0176}&{ - 0.9997} \\ {0.1516}&{0.9883}&{0.0147} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0233} \\ {0.0663} \\ {0.0262} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.2155} \\ {y:2.0061} \end{array} $
    4 $ \left[ {\begin{array}{*{20}{c}} {0.9904}&{ - 0.1377}&{ - 0.0141} \\ { - 0.0115}&{0.0199}&{ - 0.9997} \\ {0.1379}&{0.9903}&{0.0182} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0219} \\ {0.0636} \\ {0.0244} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.0629} \\ {y:4.1338} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.9883}&{ - 0.1512}&{ - 0.0176} \\ { - 0.0151}&{0.0183}&{ - 0.9997} \\ {0.1515}&{0.9883}&{0.0158} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0219} \\ {0.0635} \\ {0.0244} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.0246} \\ {y:1.9887} \end{array} $
    5 $ \left[ {\begin{array}{*{20}{c}} {0.9905}&{ - 0.1374}&{ - 0.0092} \\ { - 0.0104}&{0.0079}&{ - 0.9999} \\ {0.1373}&{0.9905}&{0.0093} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0130} \\ {0.0569} \\ {0.0258} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.4974} \\ {y:4.9327} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.9881}&{ - 0.1537}&{ - 0.0057} \\ { - 0.0027}&{0.0195}&{ - 0.9998} \\ {0.1538}&{0.9879}&{0.0189} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0130} \\ {0.0569} \\ {0.0257} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.3178} \\ {y:2.0198} \end{array} $
    Average reprojection error $ \begin{array}{*{20}{c}} {x:3.6242} \\ {y:4.3629} \end{array} $ Average reprojection error $ \begin{array}{*{20}{c}} {x:2.3080} \\ {y:2.0374} \end{array} $
    下载: 导出CSV

    表  4  红外系统实验结果

    Table  4.   Experimental results of infrared imaging system

    No. Rotation matrix Rlc (c) Translation vector tlc (c) Reprojection error (c)/pixel Rotation matrix Rlc (d) Translation vector tlc(d) Reprojection error (d)/pixel
    1 $ \left[ {\begin{array}{*{20}{c}} {0.6866}&{0.7269}&{0.0072} \\ {0.0169}&{ - 0.0060}&{ - 0.9998} \\ { - 0.7268}&{0.6866}&{ - 0.0164} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0395} \\ {0.0140} \\ {0.0235} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.0472} \\ {y:3.0035} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.6358}&{0.7717}&{0.0134} \\ {0.1654}&{ - 0.1193}&{ - 0.9789} \\ { - 0.7539}&{0.6247}&{ - 0.2034} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0395} \\ {0.0140} \\ {0.0235} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.4367} \\ {y:2.1505} \end{array} $
    2 $ \left[ {\begin{array}{*{20}{c}} {0.6701}&{0.7339}&{ - 0.0379} \\ {0.0534}&{0.0342}&{ - 0.9979} \\ { - 0.0503}&{ - 0.0796}&{ - 1.0010} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.2037} \\ { - 0.3452} \\ { - 0.0601} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.5839} \\ {y:4.3602} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.6327}&{0.7743}&{0.0100} \\ {0.1600}&{ - 0.1181}&{ - 0.9800} \\ { - 0.7577}&{0.6216}&{ - 0.1986} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0486} \\ {0.0016} \\ {0.0297} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.6007} \\ {y:2.2385} \end{array} $
    3 $ \left[ {\begin{array}{*{20}{c}} {0.6765}&{0.7363}&{0.0092} \\ {0.0168}&{ - 0.0029}&{ - 0.9998} \\ { - 0.7362}&{0.6766}&{ - 0.0143} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0803} \\ { - 0.0819} \\ {0.0745} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.3312} \\ {y:3.3278} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.6229}&{0.7822}&{0.0100} \\ {0.1317}&{ - 0.0922}&{ - 0.9870} \\ { - 0.7711}&{0.6161}&{ - 0.1604} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0803} \\ { - 0.0819} \\ {0.0745} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.8646} \\ {y:2.1971} \end{array} $
    4 $ \left[ {\begin{array}{*{20}{c}} {0.6776}&{0.7354}&{0.0112} \\ {0.0343}&{ - 0.0163}&{ - 0.9992} \\ { - 0.7346}&{0.6775}&{ - 0.0363} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.2447} \\ { - 0.0381} \\ {0.0815} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.2049} \\ {y:3.1438} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.5524}&{0.8332}&{0.0249} \\ {0.1678}&{ - 0.0819}&{ - 0.9824} \\ { - 0.8165}&{0.5469}&{ - 0.1851} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.2447} \\ { - 0.0381} \\ {0.0815} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.4126} \\ {y:2.3031} \end{array} $
    5 $ \left[ {\begin{array}{*{20}{c}} {0.6686}&{0.7436}&{0.0077} \\ { - 0.0017}&{0.0118}&{ - 0.9999} \\ { - 0.7436}&{0.6685}&{0.0092} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0673} \\ {0.0042} \\ {0.0303} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:3.0379} \\ {y:3.7253} \end{array} $ $ \left[ {\begin{array}{*{20}{c}} {0.6255}&{0.7802}&{0.0075} \\ {0.1605}&{ - 0.1192}&{ - 0.9798} \\ { - 0.7635}&{0.6141}&{ - 0.1998} \end{array}} \right] $ $ \left[ {\begin{array}{*{20}{c}} { - 0.0673} \\ {0.0042} \\ {0.0303} \end{array}} \right] $ $ \begin{array}{*{20}{c}} {x:2.6443} \\ {y:2.1625} \end{array} $
    Average reprojection error $ \begin{array}{*{20}{c}} {x:3.2410} \\ {y:3.5121} \end{array} $ Average reprojection error $ \begin{array}{*{20}{c}} {x:2.5918} \\ {y:2.2103} \end{array} $
    下载: 导出CSV
  • [1] Fremont V, Bonnifait P. Extrinsic calibration between a multi-layer lidar and a camera[C]//International Conference on Multisensor Fusion and Integration for Intelligent Systems of IEEE, 2008: 214-219.
    [2] 陈东. 三维激光和单目视觉间的联合标定与数据融合[D]. 大连: 大连理工大学, 2009.

    CHEN Dong. Calibration and Data Fusion between 3D-Laser Scanner and Monocular Vision[D]. Dalian: Dalian University of Technology, 2009.
    [3] ZHOU L, DENG Z. Extrinsic calibration of a camera and a lidar based on decoupling the rotation from the translation[C]//Intelligent Vehicles Symposium of IEEE, 2012: 642-648. .
    [4] ZHOU L, LI Z, Kaess M. Automatic extrinsic calibration of a camera and a 3d lidar using line and plane correspondences[C]//International Conference on Intelligent Robots and Systems (IROS)of IEEE, 2018: 5562-5569.
    [5] Geiger A, Moosmann F, Car O , et al. Automatic camera and range sensor calibration using a single shot[C]//Proceedings of IEEE International Conference on Robotics and Automation, 2012: 3936-3943.
    [6] Horaud R, Dornaika F. Hand-eye calibration[J]. The International Journal of Robotics Research, 1995, 14(3): 195-210.
    [7] Ishikawa R, Oishi T, Ikeuchi K. Lidar and camera calibration using motions estimated by sensor fusion odometry[C]//International Conference on Intelligent Robots and Systems(IROS) of IEEE, 2018: 7342-7349.
    [8] Schneider N, Piewak F, Stiller C, et al. RegNet: multimodal sensor registration using deep neural networks[C]//Intelligent Vehicles Symposium (IV) of IEEE, 2017: 1803-1810.
    [9] Iyer G, Ram R K, Murthy J K, et al. CalibNet: geometrically supervised extrinsic calibration using 3D spatial transformer networks[C]// International Conference on Intelligent Robots and Systems (IROS) of IEEE, 2018: 1110-1117.
    [10] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
  • 加载中
图(6) / 表(4)
计量
  • 文章访问数:  451
  • HTML全文浏览量:  73
  • PDF下载量:  57
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-03-12
  • 修回日期:  2022-04-19
  • 刊出日期:  2022-12-20

目录

    /

    返回文章
    返回