Advanced Search
中文
Volume 44 Issue 7
Jul.  2022
Turn off MathJax
Article Contents
Abstract
References
LI Yunhong, LIU Yudong, SU Xueping, LUO Xuemin, YAO Lan. Review of Infrared and Visible Image Registration[J]. Infrared Technology , 2022, 44(7): 641-651.
Citation: LI Yunhong, LIU Yudong, SU Xueping, LUO Xuemin, YAO Lan. Review of Infrared and Visible Image Registration[J]. Infrared Technology , 2022, 44(7): 641-651.
shu

Review of Infrared and Visible Image Registration

  • School of Electronics and Information, Xi'an Polytechnic University, Xi'an 710048, China

More Information
  • Received Date: April 12, 2022
  • Revised Date: May 23, 2022
    Graphical Abstract
    • Abstract
  • Multi-modal image registration can provide richer and more comprehensive information than single-modal image registration. Among them, infrared and visible image registration, which is a common multi-modal form of registration, has important application value in fields such as electric power, remote sensing, military, and face recognition. In this paper, the correlation technique of infrared and visible image registration is introduced, and the existing difficulties and challenges involved in registration are analyzed. Subsequently, the advantages and disadvantages of different registration methods are evaluated in detail the three types based on area, feature, and deep learning, and a practical application of infrared and visible image registration technology is presented. Finally, the future development trend of infrared and visible image registration is discussed.
    • FullText(HTML)
    • References (53)
  • [1]
    LI Y, YU F Y, CAI Q, et al. Image fusion of fault detection in power system based on deep learning[J]. Cluster Computing the Journal of Networks Software Tools and Applications, 2019, 22(4): 9435-9443.
    [2]
    MA J, ZHAO J, MA Y, et al. Non-rigid visible and infrared face registration via regularized Gaussian fields criterion[J]. Pattern Recognition, 2015, 48(3): 772-784. DOI: 10.1016/j.patcog.2014.09.005
    [3]
    韩静, 柏连发, 张毅, 等. 基于改进配准测度的红外与可见光图像配准[J]. 红外技术, 2011, 33(5): 271-274. DOI: 10.3969/j.issn.1001-8891.2011.05.006

    HAN J, BO L F, ZHANG Y, et al. A registration algorithm of infrared and visible image based on revised measure function[J]. Infrared Technology, 2011, 33(5): 271-274. DOI: 10.3969/j.issn.1001-8891.2011.05.006
    [4]
    Sarvaiya J N, Patnaik S, Bombaywala S. Image registration by template matching using normalized cross-correlation[C]//2009 International Conference on Advances in Computing, Control, and Telecommunication Technologies. IEEE, 2009: 819-822.
    [5]
    YANG Z, SHEN G, WANG W, et al. Spatial-spectral cross correlation for reliable multispectral image registration[C]//2009 IEEE Applied Imagery Pattern Recognition Workshop (AIPR 2009), 2009: 1-8.
    [6]
    MA J, JIANG X, FAN A, et al. Image matching from handcrafted to deep features: A survey[J]. International Journal of Computer Vision, 2021, 129(1): 23-79. DOI: 10.1007/s11263-020-01359-2
    [7]
    Stone H S, Tao B, McGuire M. Analysis of image registration noise due to rotationally dependent aliasing[J]. Journal of Visual Communication and Image Representation, 2003, 14(2): 114-135. DOI: 10.1016/S1047-3203(03)00002-6
    [8]
    LI Z, YANG J, LI M, et al. Estimation of large scalings in images based on multilayer pseudopolar fractional Fourier transform[J]. Mathematical Problems in Engineering, 2013, 2013: 179489.
    [9]
    DONG Y, JIAO W, LONG T, et al. An extension of phase correlation-based image registration to estimate similarity transform using multiple polar Fourier transform[J]. Remote Sensing, 2018, 10(11): 1719. DOI: 10.3390/rs10111719
    [10]
    Fujisawa T, Ikehara M. High-accuracy image rotation and scale estimation using radon transform and sub-pixel shift estimation[J]. IEEE Access, 2019, 7: 22719-22728. DOI: 10.1109/ACCESS.2019.2899390
    [11]
    Fouda Y, Ragab K. An efficient implementation of normalized cross-correlation image matching based on pyramid[C]//2013 International Joint Conference on Awareness Science and Technology & Ubi-Media Computing, IEEE, 2013: 98-103.
    [12]
    YANG M Y, QIANG Y, Rosenhahn B. A global-to-local framework for infrared and visible image sequence registration[C]//2015 IEEE Winter Conference on Applications of Computer Vision, 2015: 381-388.
    [13]
    BAI L F, HAN J, ZHANG Y, et al. Registration algorithm of infrared and visible images based on improved gradient normalized mutual information and particle swarm optimization[J]. Infrared Laser Engineering, 2012, 41(1): 248-254.
    [14]
    CHEN S J, SHEN H L, LI C, et al. Normalized total gradient: A new measure for multispectral image registration[J]. IEEE Transactions on Image Processing, 2017, 27(3): 1297-1310.
    [15]
    YANG T J, TANG Q, LI L. Nonrigid registration of medical image based on adaptive local structure tensor and normalized mutual information[J]. Journal of Applied Clinical Medical Physics, 2019, 20(6): 99-110. DOI: 10.1002/acm2.12612
    [16]
    赵洪山, 张则言. 基于文化狼群算法的电力设备红外和可见光图像配准[J]. 光学学报, 2020, 40(16): 51-64. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202016007.htm

    ZHAO H, ZHANG Z. Power equipment infrared and visible images registration based om cultural wolf pack algorithm[J]. Acta Optica Sinica, 2020, 40(16): 51-64. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202016007.htm
    [17]
    孙凤杰, 赵孟丹, 刘威, 等. 基于方向场的输电线路间隔棒匹配定位算法[J]. 中国电机工程学报, 2014, 34(1): 206-213. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGDC201401025.htm

    SUN F, ZHAO M, LIU W, et al. Spacer matching and localizetion algorithm for transmission line video image based on directional field[J]. Proceedings of the CSEE, 2014, 34(1): 206-213. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGDC201401025.htm
    [18]
    刘刚, 周珩, 梁晓庚, 等. 非下采样轮廓波域红外与可见光图像配准算法[J]. 计算机科学, 2016, 43(11): 313-316. DOI: 10.11896/j.issn.1002-137X.2016.11.061

    LIU G, ZHOU H, LIANG X, et al. Image registration algorithm for infrared and visible light based on non-subsamoled Contourlet transform[J]. Computer Science, 2016, 43(11): 313-316. DOI: 10.11896/j.issn.1002-137X.2016.11.061
    [19]
    吴延海, 张程, 张烨. 基于梯度信息和区域互信息的图像配准[J]. 广西大学学报: 自然科学版, 2017, 42(2): 720-727. https://www.cnki.com.cn/Article/CJFDTOTAL-GXKZ201702040.htm

    WU Y, ZHANG C, ZHANG Y. Image Registration based on gradient and regional mutual information[J]. Journal of Guangxi University: Natural Science Edition, 2017, 42(2): 720-727. https://www.cnki.com.cn/Article/CJFDTOTAL-GXKZ201702040.htm
    [20]
    ZHUANG Y, GAO K, MIU X, et al. Infrared and visual image registration based on mutual information with a combined particle swarm optimization–Powell search algorithm[J]. Optik, 2016, 127(1): 188-191. DOI: 10.1016/j.ijleo.2015.09.199
    [21]
    LI Y, WANG J, YAO K. Modified phase correlation algorithm for image registration based on pyramid[J]. Alexandria Engineering Journal, 2022, 61(1): 709-718. DOI: 10.1016/j.aej.2021.05.004
    [22]
    Morel J M, YU G. ASIFT: A new framework for fully affine invariant image comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2): 438-469. DOI: 10.1137/080732730
    [23]
    YANG N, YANG Y, LI P, et al. Research on infrared and visible image registration of substation equipment based on Multi-scale Retinex and ASIFT features[C]//Sixth International Workshop on Pattern Recognition. International Society for Optics and Photonics, 2021, 11913: 1191303.
    [24]
    ZENG Q, ADU J, LIU J, et al. Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT[J]. Journal of Real-Time Image Processing, 2020, 17(5): 1103-1115. DOI: 10.1007/s11554-019-00858-x
    [25]
    JIANG Q, LIU Y, YAN Y, et al. A contour angle orientation for power equipment infrared and visible image registration[J]. IEEE Transactions on Power Delivery, 2020, 36(4): 2559-2569.
    [26]
    李伟, 王军, 俞跃. 基于可见光匹配矩阵的电气部件故障红外自动识别算法[J]. 红外技术, 2019, 41(11): 1047-1056. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201911009.htm

    LI W, WANG J, YU Y. An infrared automatic fault recognition method for electrical parts based on visible matching matrix[J]. Infrared Technology, 2019, 41(11): 1047-1056. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201911009.htm
    [27]
    CHEN Y, DAI J, MAO X, et al. Image registration betwe en visible and infrared images for electrical equipment inspection robots based on quadrilateral features[C]// 2nd International Conference on Robotics and Automation Engineering (ICRAE), 2017: 126-130.
    [28]
    戴进墩, 刘亚东, 毛先胤, 等. 基于NSCT域FAST角点检测的电气设备红外与可见光图像配准[J]. 电测与仪表, 2019, 56(1): 108-114. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201901018.htm

    DAI J, LIU Y, MAO X, et al. Registration based on NSCT-domain FAST corner detection for infrared and visible images of electrical equipment[J]. Electrical Measurement & Instrumentation, 2019, 56(1): 108-114. https://www.cnki.com.cn/Article/CJFDTOTAL-DCYQ201901018.htm
    [29]
    江泽涛, 刘小艳, 王琦. 基于显著性和ORB的红外和可见光图像配准算法[J]. 激光与红外, 2019, 49(2): 251-256. DOI: 10.3969/j.issn.1001-5078.2019.02.022

    JIANG Z, LIU X, WANG Q. Visible and infrared image registration algorithm based on saliency and ORB[J]. Laser & Infrared, 2019, 49(2): 251-256. DOI: 10.3969/j.issn.1001-5078.2019.02.022
    [30]
    占祥慧, 徐智勇, 张建林. 结合滚动引导滤波和相位信息的红外与可见光图像配准[J]. 半导体光电, 2021, 42(5): 726-732. https://www.cnki.com.cn/Article/CJFDTOTAL-BDTG202105023.htm

    ZHANG X, XU Z, ZHANG J. Infrared and visible images registration using rolling guided filter and phase information[J]. Semiconductor Optoelectronics, 2021, 42(5): 726-732. https://www.cnki.com.cn/Article/CJFDTOTAL-BDTG202105023.htm
    [31]
    LI Q, HAN G, LIU P, et al. An infrared-visible image registration method based on the constrained point feature[J]. Sensors, 2021, 21(4): 1188. DOI: 10.3390/s21041188
    [32]
    CHEN X, LIU L, ZHANG J, et al. Registration of multimodal images with edge features and scale invariant PIIFD[J]. Infrared Physics & Technology, 2020, 111: 103549.
    [33]
    LIU X, LI J B, PAN J S, et al. An advanced gradient texture feature descriptor based on phase information for infrared and visible image matching[J]. Multimedia Tools and Applications, 2021, 80(11): 16491-16511. DOI: 10.1007/s11042-020-10213-z
    [34]
    李云红, 罗雪敏, 苏雪平, 等. 基于改进CSS的电力设备红外与可见光图像配准[J/OL]. 激光与光电子学进展: 1-14. [2021-9-9]. http://www.cnki.com.cn/Article/CJFDTotal-JGDJ20210909004.htm.

    LI Y, LUO X, SU X, et al. Registration method for power equipment infrared and visible images based on improved CSS[J/OL]. Laser & Optoelectronics Progress: 1-14. [2021-9-9]. http://www.cnki.com.cn/ Article/CJFDTotal-JGDJ20210909004.htm.
    [35]
    CHENG T, GU J, ZHANG X, et al. Multimodal image registration for power equipment using clifford algebraic geometric invariance[J]. Energy Reports, 2022, 8: 1078-1086. DOI: 10.1016/j.egyr.2022.02.192
    [36]
    陈亮, 周孟哲, 陈禾. 一种结合边缘区域和互相关的图像配准方法[J]. 北京理工大学学报, 2016, 36(3): 320-325. https://www.cnki.com.cn/Article/CJFDTOTAL-BJLG201603018.htm

    CHEN L, ZHOU M, CHEN H. A method for image registration combined by edge region and cross correlation[J]. Transactions of Beijing Institute of Technology, 2016, 36(3): 320-325. https://www.cnki.com.cn/Article/CJFDTOTAL-BJLG201603018.htm
    [37]
    LIU G, LIU Z, LIU S. et al. Registration of infrared and visible light image based on visual saliency and scale invariant feature transform[J]. J. Image Video Proc. , 2018: 45. https://doi.org/10.1186/s13640-018-0283-9.
    [38]
    廉蔺, 李国辉, 张军, 等. 基于边缘最优映射的红外和可见光图像自动配准算法[J]. 自动化学报, 2012, 38(4): 570-581. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO201204009.htm

    LIAN L, LI G, ZHANG J, et al. An automatic registration algorithm of infrared and visible images based on optimal mapping of edges[J]. Acta Automatica Sinica, 2012, 38(4): 570-581. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO201204009.htm
    [39]
    MA J, ZHAO J, MA Y, et al. Non-rigid visible and infrared face registration via regularized Gaussian fields criterion[J]. Pattern Recognition, 2015, 48(3): 772-784. DOI: 10.1016/j.patcog.2014.09.005
    [40]
    李巍, 董明利, 吕乃光, 等. 基于T分布混合模型的多光谱人脸图像配准[J]. 光学学报, 2019, 39(7): 56-66. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201907007.htm

    LI W, DONG M L, LYU N G, et al. Multispectral face image registration based on t-distribution mixture model[J]. Acta Optica Sinica, 2019, 39(7): 56-66. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201907007.htm
    [41]
    MIN C, GU Y, LI Y, et al. Non-rigid infrared and visible image registration by enhanced affine transformation[J]. Pattern Recognition, 2020, 106: 107377. DOI: 10.1016/j.patcog.2020.107377
    [42]
    MIN C, GU Y, YANG F, et al. Non-rigid registration for infrared and visible images via Gaussian weighted shape context and enhanced affine transformation[J]. IEEE Access, 2020, 8: 42562-42575. DOI: 10.1109/ACCESS.2020.2976767
    [43]
    ZHAO Z, ZHAO L, QI Y, et al. Infrared and visible image registration based on hypercolumns[C]//CCF Chinese Conference on Computer Vision, 2017: 529-539.
    [44]
    WEI Z, JUNG C, SU C. RegiNet: Gradient guided multispectral image registration using convolutional neural networks[J]. Neurocomputing, 2020, 415: 193-200. DOI: 10.1016/j.neucom.2020.07.066
    [45]
    WANG L, GAO C, ZHAO Y, et al. Infrared and visible image registration using transformer adversarial network[C]// 25th IEEE International Conference on Image Processing (ICIP). IEEE, 2018: 1248-1252.
    [46]
    Kumari K, Krishnamurthi G. GAN-based End-to-End Unsupervised Image Registration for RGB-Infrared Image[C]// 3rd International Conference on Intelligent Autonomous Systems (ICoIAS). IEEE, 2020: 62-66.
    [47]
    MAO Y, HE Z. Dual-Y network: infrared-visible image patches matching via semi-supervised transfer learning[J]. Applied Intelligence, 2021, 51(4): 2188-2197. DOI: 10.1007/s10489-020-01996-7
    [48]
    杨冰超, 王鹏, 李晓艳, 等. 基于模态转换结合鲁棒特征的红外图像和可见光图像配准[J]. 激光与光电子学进展, 2022, 59(4): 180-189. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202204017.htm

    YANG B, WANG P, LI X, et al. Infrared and visible light image registration based on model conversion and robust features[J]. Laser & Optoelectronics Progress, 2022, 59(4): 180-189. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202204017.htm
    [49]
    YU K, MA J, HU F, et al. A grayscale weight with window algorithm for infrared and visible image registration[J]. Infrared Physics & Technology, 2019, 99: 178-186.
    [50]
    LUO W, HAO X, XU C, et al. Coarse-to-fine registration for infrared and visible images of power grid[C]// 4th International Conference on Systems and Informatics (ICSAI), 2017: 1181-1185.
    [51]
    姜骞, 刘亚东, 方健, 等. 基于轮廓特征的电力设备红外和可见光图像配准方法[J]. 仪器仪表学报, 2020, 41(11): 252-260. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB202011028.htm

    JIANG Q, LIU Y D, FANG J, et al. Registration method for power equipment infrared and visible images based on contour feature[J]. Chinese Journal of Scientific Instrument, 2020, 41(11): 252-260. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB202011028.htm
    [52]
    马露凡, 罗凤, 严江鹏, 等. 深度医学图像配准研究进展: 迈向无监督学习[J]. 中国图象图形学报, 2021, 26(9): 2037-2057. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202109002.htm

    MA L F, LUO F, YAN J P, et al. Deep-learning based medical image registration pathway: towards unsupervised learning[J]. Journal of Image and Graphics, 2021, 26(9): 2037-2057. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGTB202109002.htm
    [53]
    刘欢, 肖根福, 欧阳春娟, 等. 初-精结合和多特征融合的多源遥感图像配准[J]. 遥感信息, 2018, 33(6): 61-70. DOI: 10.3969/j.issn.1000-3177.2018.06.009

    LIU H, XIAO G F, OUYANG C J, et al. Initial-fine combination and multi-feature fusion for multi-source remote sensing image registration[J]. Remote Sensing Information, 2018, 33(6): 61-70. DOI: 10.3969/j.issn.1000-3177.2018.06.009
    • Related Articles

  • Cited by

    Periodical cited type(5)

    1. 熊展雄，彭向前. 基于强边缘的空间离焦模糊复原算法. 自动化与仪表. 2024(06): 86-90+106 .
    2. 李立，易诗，刘茜，程兴豪，王铖. 基于密集残差生成对抗网络的红外图像去模糊. 红外技术. 2024(06): 663-671 . 本站查看
    3. 刘怀广，董渊，汤勃，黄千稳. 焦炭光学组织离焦显微图像去模糊算法研究. 智能计算机与应用. 2023(09): 25-32 .
    4. 宋伟. 一种基于LOG先验信息的图像去模糊方法研究. 广东通信技术. 2023(11): 55-57+79 .
    5. 刘玉雯，吴玲玲，聂亮，陈靖. 光学合成孔径图像超分辨率重建技术研究. 计算机测量与控制. 2022(02): 160-165 .

    Other cited types(0)

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by(5)
    Related

    Copyright © 《Infrared Technology》Editorial Office京ICP备000000000号-1

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return