Automotive Infrared and Visible Light Image Registration Method
-
摘要:
为了提高车辆视觉感知能力，针对交通场景运用提出一种改进的轮廓角方向（contour angle orientation，CAO）算法用于实现红外与可见光图像配准。通过模拟不同的交通场景，对成熟算法进行性能检测对比，选出CAO算法这一优势算法，并对其粗匹配参数和图像预处理图像缩放程序做了改进。实验表明，改进后的CAO算法细匹配更精准，马赛克拼接图拼接处衔接更加自然，线条更加顺滑，效果更好。与原来CAO算法相比，改进后的算法均方根误差值RMSE下降3.29%，查准率Precision提高2.13%，平均运算耗时减少0.11 s，在配准精度和配准实时性方面均证明了算法的改进效果。
Abstract:To enhance the visual perception of vehicles, an improved contour angle orientation (CAO) algorithm is proposed for the registration of infrared and visible light images in traffic scenes. By simulating different traffic scenarios, a performance comparison was conducted among mature algorithms to select the superior CAO algorithm. Subsequently, improvements were made to the coarse matching parameters and image preprocessing scaling procedure. Experiments demonstrate that the refined CAO algorithm achieves more precise fine matching, thus resulting in mosaic stitching with smoother transitions and lines and yielding better results. Compared with the original CAO algorithm, the improved version reduces the RMSE value by 3.29%, increases the precision value by 2.13%, and decreases the average computation time by 0.11 s, thereby demonstrating improvements in both registration accuracy and real-time performance.
-
-
![]()
图 2 改进前后的粗匹配示意图
Figure 2. Schematic diagram of rough matching before and after improvement
![]()
图 3 交通场景分类图例：(a)~(e)分别是非机动车场景、行人场景、机动车场景、道路岔口场景、混合交通场景
Figure 3. Traffic scenario classification legend: (a) to (e) are respectively non-motor vehicle scene, pedestrian scene, motor vehicle scene, road fork scene, and mixed traffic scene
![]()
图 6 各场景图像配准融合结果：(a) 行人场景；(b) 非机动车场景；(c) 机动车场景；(d) 道路岔口场景；(e) 混合交通场景
Figure 6. Image registration fusion results of each scene: (a) Pedestrain scene; (b) Non-motorized vehicle scene; (c) Motorized vehicle scene (d) Road intersection scene; (e) Mixed traffic scene
![]()
图 7 五种算法在各场景中的RMSE值和precision值
注：PS为行人场景、NMVS非机动车场景、MVS为机动车场景、RIS为道路岔口场景、MTS为混合交通场景
Figure 7. RMSE values and precision values of the five algorithms in each scenario
Note: PS is pedestrain scene; NMVS is non-motorized vehicle scene; MVS is motorized vehicle scene; RIS is road intersection scene; MTS is mixed traffic scene
![]()
图 10 边缘特征提取算法和改进前后CAO算法在复杂视角偏差组中性能表现
Figure 10. Performance of edge feature extraction algorithm and CAO algorithm before and after improvement in complex angle of view deviation groups
![]()
图 11 边缘特征提取算法和改进前后CAO算法在水平偏差、纵向偏差、无明显视角偏差组中性能表现
Figure 11. Performance of edge feature extraction algorithm and CAO algorithm before and after improvement in horizontal deviation, longitudinal deviation and no obvious angle of view deviation groups
![]()
图 12 算法改进前后运行耗时对比
Figure 12. Comparison of running time before and after algorithm improvement
表 1 按场景分类图像对数量
Table 1 Classifies the number of image pairs by scene
Scene classification Number of image pairs Pedestrian 5 Non-motorized vehicle 2 Motorized vehicle 2 Road intersection 5 Mixed traffic 6 
下载: 导出CSV
表 2 按视角偏差分类图像对数量
Table 2 Categorizes the number of image pairs according to viewpoint deviation
Classification of angle of view deviation Horizontal angle of view deviation Vertical angle of view deviation Complex angle of view deviation No significant angle of view deviation Number of image pairs 7 2 5 6
下载: 导出CSV
表 3 实验器材信息
Table 3 Information of the experimental equipment
Experimental equipment Related parameters Visible light camera Hikvision camera, resolution: 2048×1536, Focal length: 12 mm Infrared camera FLIR infrared camera, resolution: 640×480, Focal length: 16 mm Test platform AMD 5600X processor, 16 G memory
下载: 导出CSV
表 4 边缘特征提取算法和改进前后CAO算法的RMSE平均值与Precision平均值
Table 4 RMSE and precision average values of edge feature extraction algorithm and CAO algorithm before and after improvement
EFE CAO Improved CAO RMSE mean value 10.70 7.90 7.64 Precision mean value 0.70 0.94 0.96
下载: 导出CSV
表 5 算法改进前后在各视角偏移中的平均运行耗时
Table 5 The average running time of each angle of view offset before and after algorithm improvement
Evaluation metrics Comparison algorithm Horizontal deviation Longitudinal deviation Complex deviation No deviation Average runtime/s Original algorithm 4.58 4.79 4.63 5.14 Improved algorithm 4.50 4.24 4.59 5.07
下载: 导出CSV
-
[1] HONG R, XIANG C L, LIU H, et al. Visualizing the knowledge structure and research evolution of infrared detection technology studies[J]. Information, 2019, 10(7): 227. DOI: 10.3390/info10070227
[2] LI S T, KANG X D, FANG L Y, et al. Pixel-level image fusion: a survey of the state of the art [J]. Information Fusion, 2017, 33: 100-112. DOI: 10.1016/j.inffus.2016.05.004
[3] MA J Y, MA Y, LI C. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
[4] Castillo J C, Fernandez-Caballero A, Serrano-Cuerda J, et al. Smart environment architecture for robust people detection by infrared and visible video fusion[J]. Journal of Ambient Intelligence and Humanized Computing, 2017, 8(2): 223-237. DOI: 10.1007/s12652-016-0429-5
[5] Fendri E, Boukhriss R R, Hammami M. Fusion of thermal infrared and visible spectra for robust moving object detection[J]. Pattern Analysis and Applications, 2017, 20(4): 907-926. DOI: 10.1007/s10044-017-0621-z
[6] Apatean A, Rogozan A, Bensrhair A. Visible-infrared fusion schemes for road obstacle classification[J]. Transportation Research Part C-Emerging Technologies, 2013, 35: 180-192. DOI: 10.1016/j.trc.2013.07.003
[7] DONG Yakui, FEI Cheng, ZHAO Guopeng, et al. Registration method for infrared and visible image of sea surface vessels based on contour feature[J]. Heliyon, 2023, 9(3): 14166-14176. DOI: 10.1016/j.heliyon.2023.e14166
[8] Kumar K S, Kavitha G, Subramanian R, et al. MATLAB-A Ubiquitous Tool for the Practical Engineer[M]. Croatia: InTech, 2011: 307-326.
[9] 李大华, 郑文鹏, 李璇, 等. 基于局部归一化的电力设备红外与可见光图像配准[J/OL]. 激光与光电子学进展, 2025, 62(6): 0637007. LI Dahua, ZHENG Wengpeng, LI Xuan, et al. Infrared and visible image registration of power equipment based on local normalization[J]. Laser & Optoelectronics Progress, 2025, 62(6) : 0637007.
[10] 程国华, 王阿川, 陈舒畅, 等. 多源遥感影像高精度自动配准方法研究[J]. 液晶与显示, 2016, 31(6): 604-612. CHEN Guohua, WANG Achuan, CHEN Shuchang, et al. High accuracy-automatic registration method research on multi-source remote sensing image[J]. Liquid Crystals and Displays, 2016, 31(6): 604-612.
[11] 谭东杰, 张安. 方向相关与互信息加权组合多模图像配准方法[J]. 红外与激光工程, 2013, 42(3): 836-841. DOI: 10.3969/j.issn.1007-2276.2013.03.067 TAN Dongjie, ZHANG an, Multi-model image registration based on weighted orientation correlation and mutual information[J]. Infrared and Laser Engineering, 2013, 42(3): 836-841. DOI: 10.3969/j.issn.1007-2276.2013.03.067
[12] 李云红, 刘宇栋, 苏雪平, 等. 红外与可见光图像配准技术研究综述[J]. 红外技术, 2022, 44(7): 641-651. http://hwjs.nvir.cn/article/id/77ef812e-5018-435f-a023-771b550bedc7 LI Yunhong, LIU Yudong, SU Xueping, et al. Review of infrared and visible image registration[J]. Infrared Technology, 2022, 44(7): 641-651. http://hwjs.nvir.cn/article/id/77ef812e-5018-435f-a023-771b550bedc7
[13] 吴延海, 张程, 张烨. 基于梯度信息和区域互信息的图像配准[J]. 广西大学学报: 自然科学版, 2017, 42(2): 720-727. WU Yanhai, ZHANG Cheng, ZHANG Ye. Image registration based on gradient and regional mutual information[J]. Journal of Guangxi University(Natural Science Edition), 2017, 42(2): 720-727.
[14] 孙凤杰, 赵孟丹, 刘威, 等. 基于方向场的输电线路间隔棒匹配定位算法[J]. 中国电机工程学报, 2014, 34(1): 206-213. SUN Fengjie, ZHAO Mengdan, LIU Wei, et al. Spacer matching and localization algorithm for transmission line video images based on directional field[J]. Proceedings of the CSEE, 2014, 34(1): 206-213.
[15] 刘刚, 周珩, 梁晓庚, 等. 非下采样轮廓波域红外与可见光图像配准算法[J]. 计算机科学, 2016, 43(11): 313-316. DOI: 10.11896/j.issn.1002-137X.2016.11.061 LIU Gang, ZHOU Heng, LIANG Xiaogeng, et al. Image registration algorithm for infrared and visible light based on non-subsampled contourlet transform[J]. Computer Science, 2016, 43(11): 313-316. DOI: 10.11896/j.issn.1002-137X.2016.11.061
[16] MA J, JIANG X, FAN A, et al. Image matching from handcrafted to deep features: a survey[J]. International Journal of Computer Vision, 2021, 129(1): 23-79. DOI: 10.1007/s11263-020-01359-2
[17] MOREL J M, YU G. ASIFT: A new framework for fully affine invariant image comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2): 438-469. DOI: 10.1137/080732730
[18] YANG N, YANG Y, LI P, et al. Research on infrared and visible image registration of substation equipment based on multi-scale Retinex and ASIFT features[C]//Sixth International Workshop on Pattern Recognition. International Society for Optics and Photonics, 2021, 11913: 1191303.
[19] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[C]//European Conference on Computer Vision(ECCV), 2006, 3951(1): 404-417.
[20] JIANG Q, LIU Y, YAN Y, et al. A contour angle orientation for power equipment infrared and visible image registration[J]. IEEE Transactions on Power Delivery, 2021, 36(4): 2559-2569. DOI: 10.1109/TPWRD.2020.3011962
[21] 李云红, 罗雪敏, 苏雪平, 等. 基于改进曲率尺度空间算法的电力设备红外与可见光图像配准[J]. 激光与光电子学进展, 2022, 59(12): 138-145. LI Yunhong, LUO Xuemin, SU Xueping, et al. Registration method for power equipment infrared and visible images based on improved curvature scale space algorithm[J]. Progress in Laser and Optoelectronics, 2022, 59(12): 138-145.
[22] CHENG T, GU J, ZHANG X, et al. Multimodal image registration for power equipment using clifford algebraic geometric invariance[J]. Energy Reports, 2022, 8: 1078-1086. DOI: 10.1016/j.egyr.2022.02.192
[23] 陈亮, 周孟哲, 陈禾. 一种结合边缘区域和互相关的图像配准方法[J]. 北京理工大学学报, 2016, 36(3): 320-325. CHEN Liang, ZHOU Mengzhe, CHEN He, A method for image registration combined by edge region and cross correlation[J]. Transactions of Beijing Institute of Technology, 2016, 36(3): 320-325.
[24] 付添, 邓长征, 韩欣月, 等. 基于深度学习的电力设备红外与可见光图像配准[J]. 红外技术, 2022, 44(9): 936-943. http://hwjs.nvir.cn/article/id/1f007d8f-ee0d-4cd3-b609-1084e911d70a FU Tian, DENG Changzheng, HAN Xinyue, et al. Infrared and visible image registration for power equipment based on deep learning[J]. Infrared Technology, 2022, 44(9): 936-943. http://hwjs.nvir.cn/article/id/1f007d8f-ee0d-4cd3-b609-1084e911d70a
[25] 赵俊梅, 张利平. 复杂天气和环境下交通标志图像特征匹配技术的研究[J]. 车辆与动力技术, 2022(4): 38-43. ZHAO Junmin, ZHANG Liping. Research on image feature matching technology of traffic signs in complex weather and environment[J]. Vehicle & Power Technology, 2022(4): 38-43.
[26] 姜骞, 刘亚东, 方健, 等. 基于轮廓特征的电力设备红外和可见光图像配准方法[J]. 仪器仪表学报, 2020, 41(11): 252-260. JIANG Qian, LIU Yadong, FANG Jian, et al. Registration method for power equipment infrared and visible images based on contour feature[J]. Chinese Journal of Scientific Instrument, 2020, 41(11): 252-260.
[27] 徐伯庆, 孙国强, 陈离. 数字图像缩放的原理与硬件实现[J]. 上海理工大学学报, 2002, 24(3): 222-225. DOI: 10.3969/j.issn.1007-6735.2002.03.005 XU Boqing, SUN Guoqiang, CHEN Li. The principle and hardware implementation of digital image scaling[J]. Journal of University of Shanghai for Science and Technology, 2002, 24(3): 222-225. DOI: 10.3969/j.issn.1007-6735.2002.03.005
[28] 余先川, 吕中华, 胡丹. 遥感图像配准技术综述[J]. 光学精密工程, 2013, 21(11): 2960-2972. YU Xianchuan, LV Zhonghua, HU Dan. Review of remote sensing image registration techniques[J]. Optics and Precision Engineering, 2013, 21(11): 2960-2972.
[29] 陈勇, 皮德富, 周士源, 等. 基于小波变换的红外图像融合技术研究[J]. 红外与激光工程, 2001, 30(1): 3. CHEN Yong, PI Defu, ZHOU Shiyuan, et al. Study for infrared image fusion based on wavelet transform[J]. Infrared and Laser Engineering, 2001, 30(1): 3.
[30] Rublee E, Rabaud V, Konolige K, et al. ORB: an efficient alternative to SIFT or SURF[C]//International Conference on Computer Vision, 2011, 2564-2571(DOI: 10.1109/ICCV.2011.6126544).
-
期刊类型引用(4)
1. 李宝库，柳乐，徐伟，曾文彬，胡海飞，闫锋，蔡盛. 红外系统自身热辐射导致的分布式探测距离变化分析. 红外与激光工程. 2023(03): 104-114 . 
百度学术
2. 孔祥悦，贺融，栗洋洋，彭晴晴，杨加强，杜晓宇. 基于可见光系统的杂散光评价方法研究. 激光与红外. 2023(09): 1415-1420 . 百度学术
3. 栗洋洋，彭晴晴，刘纪洲. 透射式红外光学系统的太阳杂散辐射分析. 激光与红外. 2021(08): 1025-1030 . 百度学术
4. 费国标. 一种红外成像镜头结构设计及分析. 科学技术创新. 2020(15): 188-189 . 百度学术
其他类型引用(14)
计量
- 文章访问数: 113
- HTML全文浏览量: 20
- PDF下载量: 39
- 被引次数: 18
