Advanced Search
中文
Volume 44 Issue 9
Sep.  2022
Turn off MathJax
Article Contents
Abstract
References
Related Articles
FU Tian, DENG Changzheng, HAN Xinyue, GONG Mengqing. Infrared and Visible Image Registration for Power Equipments Based on Deep Learning[J]. Infrared Technology , 2022, 44(9): 936-943.
Citation: FU Tian, DENG Changzheng, HAN Xinyue, GONG Mengqing. Infrared and Visible Image Registration for Power Equipments Based on Deep Learning[J]. Infrared Technology , 2022, 44(9): 936-943.
shu

Infrared and Visible Image Registration for Power Equipments Based on Deep Learning

  • College of Electrical and New Energy, China Three Gorges University, Yichang 443002, China

More Information
  • Received Date: December 04, 2021
  • Revised Date: February 08, 2022
    Graphical Abstract
    • Abstract
  • A registration fusion method of infrared and visible images of power equipment based on deep learning is proposed that aims at problems with difficult and long registration time of infrared and visible images of existing power equipment. In this study, feature extraction and feature matching are combined in a deep learning framework to directly learn the mapping relationship between image block pairs and matching labels for subsequent registration. In addition, a self-learning method using infrared image and its transform image to learn the mapping function is proposed to alleviate the problem of insufficient infrared image samples during training Simultaneously, transfer learning is used to reduce the training time and accelerate the network framework. The experimental results show that the performance index of this method is significantly improved compared with the other four registration algorithms. The average accuracy of this method is 89.909, which is 2.31%, 3.36%, 2.67%, and 0.82% higher than that of the other four algorithms, respectively. The average RMSE of this method is 2.521. Compared with the other four registration algorithms, the algorithm is reduced by 14.68%, 15.24%, 4.90%, and 1.04%, respectively. The average time of the algorithm is 5.625 s, which is reduced by 5.57%, 6.82%, 2.45%, and 1.75% respectively. The efficiency of infrared and visible image registration of the power equipment must be effectively improved.
    • FullText(HTML)
    • References (31)
  • [1]
    张旭, 魏娟, 赵冬梅, 等. 电网故障诊断的研究历程及展望[J]. 电网技术, 2013, 37(10): 2745-2553. https://www.cnki.com.cn/Article/CJFDTOTAL-DWJS201310011.htm

    ZHANG Xu, WEI Juan, ZHAO Dongmei, et al. Research history and prospect of power grid fault diagnosis [J]. Power Grid Technology, 2013, 37(10): 2745-2553. https://www.cnki.com.cn/Article/CJFDTOTAL-DWJS201310011.htm
    [2]
    刘晓康, 万曦, 涂文超, 等. 基于红外可见光图像配准的电力设备分割算法[J]. 计算机与现代化, 2020(8): 26-30. https://www.cnki.com.cn/Article/CJFDTOTAL-JYXH202008005.htm

    LIU Xiaokang, WAN Xi, TU Wenchao, et al. Power equipment segmentation algorithm based on infrared and visible image registration[J]. Computer and modernization, 2020(8): 26-30. https://www.cnki.com.cn/Article/CJFDTOTAL-JYXH202008005.htm
    [3]
    王鲲鹏, 徐一丹, 于起峰. 红外与可见光图像配准方法分类及现状[J]. 红外技术, 2009, 31(5): 270-274. DOI: 10.3969/j.issn.1001-8891.2009.05.007

    WANG Kunpeng, XU Yidan, YU Qifeng. Classification and status quo of infrared and visible image registration methods[J]. Infrared Technology, 2009, 31(5): 270-274. DOI: 10.3969/j.issn.1001-8891.2009.05.007
    [4]
    赵洪山, 张则言. 基于文化狼群算法的电力设备红外和可见光图像配准[J]. 光学学报, 2020, 40(16): 1-14. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202016007.htm

    ZHAO Hongshan, ZHANG Zeyan Infrared and visible image registration of power equipment based on cultural wolf swarm algorithm[J]. Acta Optica Sinica, 2020, 40(16): 1-14. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202016007.htm
    [5]
    MA J Y, JIANG X Y, FAN A X, et al. Image matching from handcrafled to deep features: A survey [J]. International Journal of Computer Vision, 2021, 129(1): 23-79. DOI: 10.1007/s11263-020-01359-2
    [6]
    徐全飞, 冯旗. 基于SURF和矩阵乘法的超大规模遥感图像亚像素配准算法研究[J]. 红外技术, 2017, 39(1): 44-52. http://hwjs.nvir.cn/article/id/hwjs201701009

    XU Quanfei, FENG Qi Research on subpixel registration algorithm of super large scale remote sensing image based on surf and matrix multiplication[J]. Infrared Technology, 2017, 39(1): 44-52. http://hwjs.nvir.cn/article/id/hwjs201701009
    [7]
    WU G R, KIM M, WANG Q, GAO Y Z, et al. Unsupervised deep feature learning for deformable registration of MR brain image[C]//Proceedings of the 16th International Conference on Medical Image Computing and Computer-Assisted Intervention, 2013: 649-656.
    [8]
    CHENG X, ZHANG L, ZHENG Y F. Deep similarity learning for multimodal medical images[J]. Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, 2018, 6(3): 248-252.
    [9]
    Simonovsky M, Gulierrez-Becker B, Mateus D, et al. A deep metric for multimodal registration[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer Assisted Intervention, 2016: 10-18.
    [10]
    Kabus S, Netsch T, Fischer B, et al. B-spline registration of 3D images with Levenberg-Marquardt optimization[C]//Proceedings of SPIE, Medical Imaging, Image Processing, 2004, 5370: 304-313.
    [11]
    TANG L S, Hamarneh G, Celler A. Co-registration of bone CT and SPECT images using mutual information[C]//IEEE International Symposium on Signal Processing and Information Technology, 2006: 116-121.
    [12]
    MIAO S, WANG Z J, LIAO R. A CNN regression approach for real-time 2D/3D registration[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1352-1363.
    [13]
    HAN Xufeng, LEUNG T, JIA Y, et al. MatchNet: Unifying feature and metric learning for patch-based matching[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 3279-3286, Doi: 10.1109/CVPR.2015.7298948.
    [14]
    Krebs J, Mansi T, Delingette H, et al. Robust non-rigid registration through agent-based action learning[C]// Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, 2017: 344-35.
    [15]
    Dosovitskiy A, Fischer P, Ilg E, et al. FlowNet: learning optical flow with convolutional networks[C]// Proceedings of IEEE International Conference on Computer Vision, 2015: 2758-2766.
    [16]
    LI H W, WANG B H, LI L. Research on the infrared and visible power-equipment image fusion for inspection robots[C]//2010 1st International Conference on Applied Robotics for the Power Industry, 2010: 1-5.
    [17]
    李寒, 王库, 刘韶军. 基于灰度冗余和SURF算法的电气设备红外和可见光图像配准[J]. 电力系统保护与控制, 2011, 39(11): 111-115, 123. https://www.cnki.com.cn/Article/CJFDTOTAL-JDQW201111021.htm

    LI Han, WANG Ku, LIU Shaojun. Infrared and visible image registration of electrical equipment based on gray redundancy and surf algorithm[J]. Power System Protection and Control, 2011, 39(11): 111-115, 123. https://www.cnki.com.cn/Article/CJFDTOTAL-JDQW201111021.htm
    [18]
    姜骞, 刘亚东, 方健, 等. 基于轮廓特征的电力设备红外和可见光图像配准方法[J]. 仪器仪表学报, 2020, 41(11): 252-260. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB202011028.htm

    JIANG Qian, LIU Yadong, FANG Jian, et al. Infrared and visible image registration method of power equipment based on contour feature[J]. Journal of Instrumentation, 2020, 41(11): 252-260. https://www.cnki.com.cn/Article/CJFDTOTAL-YQXB202011028.htm
    [19]
    李云红, 罗雪敏, 苏雪平, 等. 基于改进CSS的电力设备红外与可见光图像配准[J]. 激光与光电子学进展, 2021, 11(24): 1-15. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202212014.htm

    LI Yunhong, LUO Xuemin, SU Xueping, et al. Infrared and visible image registration of power equipment based on improved CSS [J]. Progress in Laser and Optoelectronics, 2021, 11(24): 1-15. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202212014.htm
    [20]
    Taylor S, Drummond T. Binary histogrammed intensity patches for efficient and robust matching[J]. International Journal of Computer Vision, 2011, 94(2): 241-265.
    [21]
    MA W, ZHANG J, WU Y, et al. A novel two-step registration method for remote sensing images based on deep and local features[J]. IEEE Trans Geosci Remote Sen, 2019, 57: 4834-4843.
    [22]
    YE F, SU Y, XIAO H, et al. Remote sensing image registration using convolutional neural network features[J]. IEEE Geosci. Remote Sens Lett. , 2018, 15: 232-236.
    [23]
    LIU X L, JIANG D S, WANG M N, et al. Image synthesis based multi-modal image registration framework by using deep fully convolutional networks[J]. Medical & Biological Engineering & Computing, 2019, 57(5): 1037-1048.
    [24]
    周微硕, 安博文, 赵明, 等. 基于几何不变性和局部相似特征的异源遥感图像配准算法[J]. 红外技术, 2019, 41(6): 561-571. http://hwjs.nvir.cn/article/id/hwjs201906012

    ZHOU Weishuo, AN Bowen, ZHAO Ming, et al A registration algorithm for heterogeneous remote sensing images based on geometric invariance and local similarity [J]. Infrared Technology, 2019, 41(6): 561-571. http://hwjs.nvir.cn/article/id/hwjs201906012
    [25]
    Salakhutdinov R, Hinton G E. Deep Boltzmann machines[C]// Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, 2009: 448-455.
    [26]
    杨琼楠, 马天力, 杨聪锟, 等. 基于优化采样的RANSAC图像匹配算法[J]. 激光与光电子学进展, 2020, 57(10): 259-266. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010029.htm

    YANG Qiongnan, MA Tianli, YANG Congkun, et al. RANSAC image matching algorithm based on optimized sampling[J]. Progress in Laser and Optoelectronics, 2020, 57(10): 259-266. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ202010029.htm
    [27]
    李航. 统计学习方法[M]. 北京: 清华大学出版社, 2012.

    LI Hang. Statistical Learning Method[M]. Beijing: Tsinghua University Press, 2012.
    [28]
    朱英宏, 李俊山, 汤雨. 基于CSS角点提取的红外与可见光图像匹配算法[J]. 系统工程与电子技术, 2011, 33(11): 2540-2545. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD201111037.htm

    ZHU Yinghong, LI Junshan, TANG Yu. Infrared and visible image matching algorithm based on CSS corner extraction[J]. Systems Engineering and Electronic Technology, 2011, 33(11): 2540-2545. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD201111037.htm
    [29]
    岳国华, 邢晓利. 基于卷积神经网络和校正网络相结合的遥感图像配准方法研究[J]. 计算机应用与软件, 2021, 38(11): 185-190. https://www.cnki.com.cn/Article/CJFDTOTAL-JYRJ202111031.htm

    YUE Guohua, XING Xiaoli. Research on remote sensing image registration method based on convolution neural network and correction network[J]. Computer Applications and Software, 2021, 38(11): 185-190. https://www.cnki.com.cn/Article/CJFDTOTAL-JYRJ202111031.htm
    [30]
    FU J, LI W S, XU L M. DSAGAN: A generative adversarial network based on dual-stream attention mechanism for anatomical and functional image fusion [J]. Information Sciences, 2021, 576: 484-506.
    [31]
    王少杰, 武文波, 徐其志. VGG与DoG结合的光学遥感影像精确配准方法[J]. 航天返回与遥感, 2021, 42(5): 76-84. https://www.cnki.com.cn/Article/CJFDTOTAL-HFYG202105010.htm

    WANG Shaojie, WU Wenbo, XU Qizhi. An accurate registration method of optical remote sensing images based on VGG and DoG[J]. Aerospace Return and Remote Sensing, 2021, 42(5): 76-84. https://www.cnki.com.cn/Article/CJFDTOTAL-HFYG202105010.htm
    • Related Articles

  • Related Articles

    [1]CHEN Jia, YU Chengbo, WANG Shibing, JIANG Qichao, HE Xin, ZHANG Wei. IR Image Classification and Detection of Power Equipment Based on CBAM Improvement[J]. Infrared Technology , 2025, 47(1): 72-80.
    [2]WANG Xiaodong, LYU Tongfa, BAO Mingzheng, HE Yongchun, XIN Peng, WU Tao. Infrared Image Recognition Method for Power Equipment Based on Improved YOLO v5[J]. Infrared Technology , 2024, 46(6): 722-727.
    [3]HUANG Zhihong, XIAO Jian, XU Xianyong, ZHANG Hui. Spectral Residual Transformation for Thermal Defect Detection of Power Equipment[J]. Infrared Technology , 2023, 45(8): 884-889.
    [4]LIU Yunfeng, ZHAO Hongshan, YANG Jinbiao, HAN Jinfeng, LIU Bingcong. Super Resolution Method for Power Equipment Infrared Imaging Based on Gradient Norm-ratio Prior[J]. Infrared Technology , 2023, 45(1): 40-48.
    [5]LIANG Jian, HUANG Zhihong, ZHANG Keren. Multi-scale Guided Filter and Decision Fusion for Thermal Fault Diagnosis of Power Equipment[J]. Infrared Technology , 2022, 44(12): 1344-1350.
    [6]HUANG Zhihong, HONG Feng, HUANG Wei. Shape Adaptation Low Rank Representation for Thermal Fault Diagnosis of Power Equipments[J]. Infrared Technology , 2022, 44(8): 870-874.
    [7]SU Haifeng, ZHAO Yan, WU Zejun, CHENG Bo, LYU Linfei. Refined Infrared Object Detection Model for Power Equipment Based on Improved RetinaNet[J]. Infrared Technology , 2021, 43(11): 1104-1111.
    [8]HUANG Zhihong, WU Sheng, XIAO Jian, ZHANG Keren, HUANG Wei. Thermal Fault Diagnosis of Power Equipments Based on Guided Filter[J]. Infrared Technology , 2021, 43(9): 910-915.
    [9]WANG Xiaofang, MAO Huamin. Infrared Image Segmentation Method for Power Equipment in Complex Background[J]. Infrared Technology , 2019, 41(12): 1111-1116.
    [10]CHEN Jishun, XIAO Lijun, WAN Xinyu, MAI Ruijie, QIN Huiping, LI Lei. Research on Enhancement and Segmentation of Power Equipment Infrared Heat Map in Complex Environment[J]. Infrared Technology , 2018, 40(11): 1112-1118.

  • Cited by

    Periodical cited type(6)

    1. 王龙飞，胡雨旺，张泽光，刘悦，薛常喜. 非球面光学元件的快速制造技术. 光电工程. 2024(01): 3-34 .
    2. 刘斌，王孝坤，程强，刘忠凯，李凌众，蔡梦雪，张洛嘉，李文涵，王金成，张学军. 复杂曲面光学元件高精度面形检测技术概述. 南通大学学报(自然科学版). 2024(01): 1-27 .
    3. 席建普，周卓林，罗来华，张通通，聂子凯. 自由曲面轮廓仪机架多目标优化设计. 制造技术与机床. 2024(05): 122-127 .
    4. 简恩霖，肖红军，张芷芊，杨皓桦，吴南寿，曾亚光. 谱域OCT技术在非球面镜面型测量中的应用. 现代信息科技. 2024(09): 58-61 .
    5. 叶卉，王泽华，许世炜，姜晨. 基于便携式粗糙度仪的表面轮廓测量实验平台设计. 实验技术与管理. 2024(07): 140-147 .
    6. 王兆明，栗孟娟，于秋跃，李春林，赵子跃，王经华，吕天斌，张兆健，于长锁. 两面共体非球面反射镜光轴一致性高精度测量方法研究(特邀). 红外与激光工程. 2023(09): 39-46 .

    Other cited types(2)

Catalog

    Get Citation
    PDF
    XML
    Article views (359) PDF downloads (65) Cited by(8)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return