稀疏深度特征红外图像拼接算法

胡俊伟, 王诗薇, 杨默远

胡俊伟, 王诗薇, 杨默远. 稀疏深度特征红外图像拼接算法[J]. 红外技术, 2025, 47(5): 584-590.
引用本文: 胡俊伟, 王诗薇, 杨默远. 稀疏深度特征红外图像拼接算法[J]. 红外技术, 2025, 47(5): 584-590.
HU Junwei, WANG Shiwei, YANG Moyuan. Sparse Depth Feature Infrared Image Stitching Algorithm[J]. Infrared Technology , 2025, 47(5): 584-590.
Citation: HU Junwei, WANG Shiwei, YANG Moyuan. Sparse Depth Feature Infrared Image Stitching Algorithm[J]. Infrared Technology , 2025, 47(5): 584-590.

稀疏深度特征红外图像拼接算法

基金项目: 

国防科技基础加强计划资助 2021-JCJQ-JJ-1020

详细信息
    作者简介:

    胡俊伟(1997-),男,硕士,工程师,主要研究方向为计算机视觉与图像处理。E-mail:hujunwei_zs@foxmail.com

    通讯作者:

    王诗薇(1991-),女,博士,高级工程师,主要研究方向为计算机视觉。E-mail:wangshiwei_124@163.com

  • 中图分类号: TN911.73

Sparse Depth Feature Infrared Image Stitching Algorithm

  • 摘要:

    为解决红外图像拼接过程中存在的红外特征少、特征匹配效果差等问题,本文提出稀疏深度特征红外图像拼接(Sparse Depth Feature infrared image Stitching,SDFS)算法。该算法先基于卷积神经网络提取密集深度特征图,然后从特征图中计算和描述稀疏特征点,提高特征点的提取质量;然后提出使用K-近邻搜寻法完成稀疏特征点粗匹配,再通过动态距离比策略精细化匹配结果,提升匹配精度;最后依据匹配结果计算单应矩阵进行图像投影变换,并使用自适应因子加权融合完成图像无缝融合拼接。实验结果证明该算法鲁棒性高,可有效适应不同场景的红外图像拼接,拼接准确率和显示效果都高于常用的基于SIFT、SURF特征提取的拼接算法。

    Abstract:

    To solve the problems of limited infrared features and poor feature matching in the process of infrared image stitching, this paper proposes an algorithm called sparse depth feature infrared image stitching (SDFS). The algorithm first extracts a dense depth feature map using a convolutional neural network, then calculates and describes sparse feature points from the feature map to enhance the quality of feature point extraction. Next, the K-nearest neighbor search method is used to perform coarse matching of sparse feature points, followed by the application of a dynamic distance ratio strategy to refine the matching results and improve matching accuracy. Finally, based on the matching results, a homography matrix is calculated for image projection transformation, and adaptive factor-weighted fusion is used to achieve seamless fusion and splicing of the image. Experimental results show that the algorithm exhibits high robustness and can effectively adapt to infrared image stitching in different scenes. The stitching accuracy and display effect outperform commonly used stitching algorithms based on SIFT or SURF feature extraction.

  • 图  1   SDFS总体流程图

    Figure  1.   Overall flow chart of SDFS

    图  2   稀疏深度特征提取模块结构图

    Figure  2.   Structure diagram of the sparse depth feature extraction module

    图  3   实验结果对比

    Figure  3.   Comparison of experimental results

    图  4   特征匹配对数量示意

    Figure  4.   Schematic diagram of the number of feature matching pairs

    表  1   实验结果准确率对比

    Table  1   Comparison of the accuracy of the test results %

    Methods SIFT SURF SDFS*
    Acc 76 80 88
    Err 17 14 8
    下载: 导出CSV

    表  2   程序运行时间对比

    Table  2   Comparison of program running times s

    Method SIFT SURF SDFS*
    Time 0.354 0.190 2.103
    下载: 导出CSV
  • [1] 李相迪, 黄英, 张培晴, 等. 红外成像系统及其应用[J]. 激光与红外, 2014, 44(3): 229-234.

    LI Xiangdi, HUANG Ying, ZHANG Peiqing, et al. Infrared imaging system and applications[J]. Infrared Technology, 2014, 44(3): 229-234.

    [2] 王向军, 张嘉豪, 王越. 基于位姿数据校正的遥感红外图像相似性先验拼接[J]. 红外技术, 2019, 41(12): 1091-1099. http://hwjs.nvir.cn/article/id/hwjs201912001

    WANG Xiangjun, ZHANG Jiahao, WANG Yue. Image stitching of remote sensing infrared images with similarity prior based on correction of pose data [J]. Infrared Technology, 2019, 41(12): 1091-1099. http://hwjs.nvir.cn/article/id/hwjs201912001

    [3] 周登科, 郭星辰, 史凯特, 等. 风电场无人机巡检红外叶片图像拼接算法[J]. 红外技术, 2023, 45(11): 1161-1168. http://hwjs.nvir.cn/article/id/d993d1a1-2555-46b6-a75b-8f0c155cc9fb

    ZHOU Dengke, GUO Xingchen, SHI Kaite, et al. Infrared blade image stitching algorithm for unmanned aerial vehicle inspection in wind farms[J]. Infrared Technology, 2023, 45(11): 1161-1168. http://hwjs.nvir.cn/article/id/d993d1a1-2555-46b6-a75b-8f0c155cc9fb

    [4]

    Lowe D G. Distinctive Image features from scale-invariant key points[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI: 10.1023/B:VISI.0000029664.99615.94

    [5] 卢泉, 杨振华, 黄粒峰. 改进最佳缝合线的红外图像拼接方法[J]. 红外技术, 2022, 44(6): 580-586. http://hwjs.nvir.cn/article/id/ff102dc9-5b92-41f9-8a8f-813d8d5f0c8b

    LU Quan, YANG Zhenhua, HUANG Lifeng. Improved infrared image stitching method for optimal suture [J]. Infrared Technology, 2022, 44(6): 580-586. http://hwjs.nvir.cn/article/id/ff102dc9-5b92-41f9-8a8f-813d8d5f0c8b

    [6] 王昕平, 张森林, 刘妹琴, 等. 基于多尺度图像融合和SIFT特征的水下图像拼接研究[J]. 计算机应用与软件, 2021, 38(5): 213-217, 230. DOI: 10.3969/j.issn.1000-386x.2021.05.034

    WANG Xinping, ZHANG Senlin, LIU Meiqin, et al. Research on underwater image stitching based on multi-scale image fusion and SIFT features[J]. Computer Applications and Software, 2021, 38(5): 213-217, 230. DOI: 10.3969/j.issn.1000-386x.2021.05.034

    [7]

    Rosten E, Porter R, Drummond T. Faster and better: a machine learning approach to corner detection[J]. IEEE Trans Pattern Anal Mach Intell, 2008, 32(1): 105-119.

    [8] 邓秋菊, 王宁, 陆阳. 基于全局和局部特征的红外图像快速拼接研究[J]. 激光杂志, 2023, 44(10): 84-88.

    DENG Qiuju, WANG Ning, LU Yang. Research on fast infrared image stitching based on global and local features[J]. Laser Journal, 2023, 44(10): 84-88

    [9]

    Rublee E, Rabaud V, Konolige K, et al. ORB: An efficient alternative to SIFT or SURF[C]//ICCV, 2011, 11(1): 2564-2571.

    [10] 彭畅, 李广泽, 张晓阳, 等. 基于改进的ORB算法的红外遥感图像拼接研究[J]. 控制工程, 2020, 27(8): 1332-1336.

    PENG Chang, LI Guangze, ZHANG Xiaoyang, et al. Research on infrared remote sensing image stitching based on improved ORB algorithm [J]. Control Engineering, 2020, 27(8): 1332-1336

    [11]

    Bay H, Tuytelaars T, Gool L V. SURF: speeded up robust features[C]//European Conference on Computer Vision, 2006: 404-417.

    [12] 车畅, 兰文宝. Canny算法和中值滤波法的红外全景图像拼接[J]. 激光杂志, 2020, 41(5): 109-113.

    CHE Chang, LAN Wenbao. Canny algorithm and median filtering method for infrared panoramic image stitching [J]. Laser Journal, 2020, 41(5): 109-113

    [13] 蒋敏, 蒋品群, 宋树祥, 等. 基于点线配准和超像素分割的图像拼接算法[J]. 计算机应用与软件, 2023, 40(10): 250-254, 267.

    JIANG Min, JIANG Pinqun, SONG Shuxiang, et al. Image stitching algorithm based on point line registration and superpixel segmentation[J]. Computer Applications and Software, 2023, 40(10): 250-254, 267.

    [14] 杨利春, 田彬, 党建武. 基于深度学习的图像拼接算法研究综述[J]. 计算机应用研究, 2024, 41(7): 1-11.

    YANG Lichun, TIAN Bin, DANG Jianwu. A review of image stitching algorithms based on deep learning [J]. Computer Application Research, 2024: 1-11.

    [15]

    YAN Ni, MEI Yupeng, XU Ling, et al. Deep learning on image stitching with multi-viewpoint images: a survey[J]. Neural Processing Letters, 2023, 55(4): 3863-3898.

    [16]

    LE Hoang, LIU Feng, ZHANG Shu, et al. Deep homography estimation for dynamic scenes[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020, 7649-7658.

    [17] 郭璠, 李小虎, 刘文韬, 等. 基于参数回归的快速全景图像拼接算法[J]. 通信学报, 2023, 44(9): 36-47.

    GUO Fan, LI Xiaohu, LIU Wentao, et al. A fast panoramic image stitching algorithm based on parameter regression [J]. Journal of Communications, 2023, 44(9): 36-47.

    [18]

    HONG Mingbo, LU Yuhang, YE Nianjin, et al. Unsupervised homography estimation with coplanarity-aware gan[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2022: 17642-17651.

    [19]

    ZHOU Hongfei, ZHU Yuhe, LV Xiaoqian, et al. Rectangular-output image stitching[C]//2023 IEEE International Conference on Image Processing (ICIP), 2023: 2800-2804.

    [20] 蓝朝桢, 卢万杰, 于君明, 等. 异源遥感影像特征匹配的深度学习算法[J]. 测绘学报, 2021, 50(2): 189-202.

    LAN Chaozhen, LU Wanjie, YU Junming, et al. Deep learning algorithm for feature matching of heterogeneous remote sensing images[J]. Journal of Surveying and Mapping, 2021, 50(2): 189-202

    [21]

    Dusmanu M, Rocco I, Pajdla T, et al. D2-Net: a trainable CNN for joint detection and description of local features[J]. CoRR, 2019, abs/1905.03561.

    [22]

    Cover Thomas M, Peter E. Hart. Nearest neighbor pattern classification[J]. IEEE Trans. Inf. Theory, 1967(13): 21-27。

    [23]

    LI Feifei, JIA Deng, KAI Li. ImageNet: constructing a large-scale image database[J]. Journal of Vision, 2010, 9(8): 1037-1037.

    [24]

    Christopher W, Omar A, N D B, et al. Megadepth: efficient coverage quantification for BigWigs and BAMs[J]. Bioinformatics, 2021, 37(18): 3014-3016.

    [25]

    Fischler M A, Bolles R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.

图(4)  /  表(2)
计量
  • 文章访问数:  68
  • HTML全文浏览量:  12
  • PDF下载量:  24
  • 被引次数: 0
出版历程
  • 收稿日期:  2024-04-28
  • 修回日期:  2024-09-22
  • 网络出版日期:  2025-05-27
  • 刊出日期:  2025-05-19

目录

    /

    返回文章
    返回
    x 关闭 永久关闭

    尊敬的专家、作者、读者:

    端午节期间因系统维护,《红外技术》网站(hwjs.nvir.cn)将于2024年6月7日20:00-6月10日关闭。关闭期间,您将暂时无法访问《红外技术》网站和登录投审稿系统,给您带来不便敬请谅解!

    预计6月11日正常恢复《红外技术》网站及投审稿系统的服务。您如有任何问题,可发送邮件至编辑部邮箱(irtek@china.com)与我们联系。

    感谢您对本刊的支持!

    《红外技术》编辑部

    2024年6月6日