| Citation: |
LI Ziqian, BAN Yanwameng, LIU Yun, HE Dong, DU Rucai. Visible and Infrared Image Matching Method Based on Multi-Scale Feature Point Extraction[J]. Infrared Technology , 2025, 47(3): 351-357.
|
A visible and infrared image matching method (VIMN) based on multiscale feature point extraction is proposed to address the issues of low matching accuracy and poor applicability, caused by significant differences in image features in visible and infrared image matching tasks. First, to enhance the ability of the VIMN to adapt to geometric image transformations, a deformable convolution layer is introduced into the feature extraction module. A spatial pyramid pooling (SPP) layer is used to complete multiscale feature fusion, considering both low- and high-level semantic information of an image. Second, a joint feature space and channel response score map are constructed on the multiscale fusion feature map to extract robust feature points. Finally, an image patch matching module uses metric learning for visible light and infrared image matching. To verify the superiority of the VIMN matching method, comparative experiments were conducted on matching experimental datasets using scale-invariant feature transform (SIFT), particle swarm optimization (PSO)-SIFT, dual disentanglement network (D2 Net), and contextual multiscale multilevel network (CMM-Net). The qualitative and quantitative results indicate that the VIMN proposed in this study has better matching performance.
| [1] |
龙雨馨, 赖文杰, 张怀元, 等. 基于梯度方向直方图的红外与可见光融合网络的损失函数[J]. 激光与光电子学进展, 2023, 60(24): 2411001.
LONG Yuxin, LAI Wenjie, ZHANG Huaiyuan, et al. Soft histogram of gradients loss: a loss function for optimization of the image fusion networks[J]. Laser & Optoelectronics Progress, 2023, 60(24): 2411001.
|
| [2] |
宁大海, 郑晟. 可见光和红外图像决策级融合目标检测算法[J]. 红外技术, 2023, 45(3): 282-291. http://hwjs.nvir.cn/article/id/5340b616-c317-4372-9776-a7c81ca2c729
NING Dahai, ZHENG Sheng. An object detection algorithm based on decision-level fusion of visible and infrared images[J]. Infrared Technology, 2023, 45(3): 282-291. http://hwjs.nvir.cn/article/id/5340b616-c317-4372-9776-a7c81ca2c729
|
| [3] |
刘雪. 基于深度学习的考场异常行为检测研究与应用[D]. 南充: 西华师范大学, 2023.
LIU Xue. Research and Application of Abnormal Behavior Detection in Exam Rooms Based on Deep Learning[D]. Nanchong: China West Normal University, 2023.
|
| [4] |
罗迪, 王从庆, 周勇军. 一种基于生成对抗网络与注意力机制的可见光和红外图像融合方法[J]. 红外技术, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
LUO Di, WANG Congqing, ZHOU Yongjun. A visible and infrared image fusion method based on generative adversarial networks and attention mechanism[J]. Infrared Technology, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
|
| [5] |
JIANG X, MA J, XIAO G, et al. A review of multimodal image matching: methods and applications[J]. Information Fusion, 2021, 73: 22-71. DOI: 10.1016/j.inffus.2021.02.012
|
| [6] |
徐永会, 杨德智, 刘芳名. 基于对数极坐标和频域率的互信息图像配准[J]. 舰船电子工程, 2022, 42(11): 86-89, 150.
XU Yonghui, YANG Dezhi, LIU Fangming. Mutual information image registration based on logarithmic polar coordinates and frequency domain rate[J]. Ship Electronic Engineering, 2022, 42(11): 86-89, 150.
|
| [7] |
CUI Z, QI W, LIU Y. A fast image template matching algorithm based on normalized cross correlation[C]//Journal of Physics: Conference Series, 2020, 1693(1): 012163.
|
| [8] |
Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. http://pdfs.semanticscholar.org/6984/591a4ecff1b6b3d9549a3a801a37acc23426.pdf
|
| [9] |
MA W, WEN Z, WU Y, et al. Remote sensing image registration with Modified SIFT and enhanced feature matching[J]. IEEE Geoscience & Remote Sensing Letters, 2016, 14(1): 3-7. http://www.onacademic.com/detail/journal_1000039770692810_af5e.html
|
| [10] |
YE Y, SHAN J, Bruzzone L, et al. Robust registration of multimodal remote sensing images based on structural similarity[J]. arXiv preprint arXiv: 2103.16871, 2021.
|
| [11] |
DeTone D, Malisiewicz T, Rabinovich A. Superpoint: self-supervised interest point detection and description[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018: 224-236.
|
| [12] |
Dusmanu M, Rocco I, Pajdla T, et al. D2-Net: a trainable cnn for joint detection and description of local features[J]. arXiv preprint arXiv: 1905.03561, 2019.
|
| [13] |
蓝朝桢, 卢万杰, 于君明. 异源遥感影像特征匹配的深度学习算法[J]. 测绘学报, 2021, 50(2): 189-202.
LAN Chaozhen, LU Wanjie, YU Junming, et al. Deep learning algorithm for feature matching of cross modality remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 189-202.
|
| [14] |
ZHANG H, LI L, NI W, et al. Explore better network framework for high-resolution optical and SAR image matching[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-18.
|
| [15] |
TIAN Y, FAN B, WU F. L2-net: Deep learning of discriminative patch descriptor in euclidean space[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 661-669.
|
| [16] |
WANG Y, YANG J, WANG L, et al. Light field image super-resolution using deformable convolution[J]. IEEE Transactions on Image Processing, 2020, 30: 1057-1071.
|
| [17] |
ZHANG L, Rusinkiewicz S. Learning to detect features in texture images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 6325-6333.
|
| [18] |
JIA X, ZHU C, LI M, et al. LLVIP: A visible-infrared paired dataset for low-light vision[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 3496-3504.
|
| [19] |
Toet A. The TNO multiband image data collection[J]. Data in Brief, 2017, 15: 249-251. http://www.xueshufan.com/publication/2757470902
|
| [20] |
Ghazali K H B, MA J. An innovative face detection based on skin color segmentation[J]. International Journal of Computer Applications, 2011, 34(2): 6-10. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=66C46CCFE38048B39749283BE539BD63?doi=10.1.1.259.1352&rep=rep1&type=pdf
|
| [1] | DAI Yueming, YANG Lufeng, TONG Xiongmin. Real-time Section State Verification Method of Energy Management System Low Voltage Equipment Based on Infrared Image and Deep Learning[J]. Infrared Technology , 2024, 46(12): 1464-1470. |
| [2] | CHEN Chaoyang, JIANG Yuanyuan. Infrared and Visible Image Fusion Based on Deep Image Decomposition[J]. Infrared Technology , 2024, 46(12): 1362-1370. |
| [3] | BAI Hao, BAI Tingzhu. Infrared Image Super-Resolution Reconstruction Algorithm Based on Deep Residual Neural Network[J]. Infrared Technology , 2024, 46(2): 176-182. |
| [4] | DUAN Jin, ZHANG Hao, SONG Jingyuan, LIU Ju. Review of Polarization Image Fusion Based on Deep Learning[J]. Infrared Technology , 2024, 46(2): 119-128. |
| [5] | FU Tian, DENG Changzheng, HAN Xinyue, GONG Mengqing. Infrared and Visible Image Registration for Power Equipments Based on Deep Learning[J]. Infrared Technology , 2022, 44(9): 936-943. |
| [6] | LI Yunhong, LIU Yudong, SU Xueping, LUO Xuemin, YAO Lan. Review of Infrared and Visible Image Registration[J]. Infrared Technology , 2022, 44(7): 641-651. |
| [7] | ZHANG Yutong, ZHAI Xuping, NIE Hong. Deep Learning Method for Action Recognition Based on Low Resolution Infrared Sensors[J]. Infrared Technology , 2022, 44(3): 286-293. |
| [8] | ZHONG Rui, YANG Li, DU Yongcheng. The Influence of Deep Transfer Learning Pre-training on Infrared Wake Image Recognition[J]. Infrared Technology , 2021, 43(10): 979-986. |
| [9] | FAN Peng, FENG Wanxing, ZHOU Ziqiang, ZHAO Chun, ZHOU Sheng, YAO Xiangyu. Application of Deep Learning in Abnormal Insulator Infrared Image Diagnosis[J]. Infrared Technology , 2021, 43(1): 51-55. |
| [10] | YANG Tao, DAI Jun, WU Zhongjian, JIN Daizhong, ZHOU Guojia. Target Recognition of Infrared Ship Based on Deep Learning[J]. Infrared Technology , 2020, 42(5): 426-433. |
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: