留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于NSCT和最小化-局部平均梯度的图像融合

杨孙运 奚峥皓 王汉东 罗晓 阚秀

杨孙运, 奚峥皓, 王汉东, 罗晓, 阚秀. 基于NSCT和最小化-局部平均梯度的图像融合[J]. 红外技术, 2021, 43(1): 13-20.
引用本文: 杨孙运, 奚峥皓, 王汉东, 罗晓, 阚秀. 基于NSCT和最小化-局部平均梯度的图像融合[J]. 红外技术, 2021, 43(1): 13-20.
YANG Sunyun, XI Zhenghao, WANG Handong, LUO Xiao, KAN Xiu. Image Fusion Based on NSCT and Minimum-Local Mean Gradient[J]. Infrared Technology , 2021, 43(1): 13-20.
Citation: YANG Sunyun, XI Zhenghao, WANG Handong, LUO Xiao, KAN Xiu. Image Fusion Based on NSCT and Minimum-Local Mean Gradient[J]. Infrared Technology , 2021, 43(1): 13-20.

基于NSCT和最小化-局部平均梯度的图像融合

基金项目: 

国家自然科学基金项目 61803255

详细信息
    作者简介:

    杨孙运(1993-)男,硕士研究生,研究方向:智能感知,智能控制,数据分析

    通讯作者:

    阚秀(1983-)女,博士,副教授。研究方向:智能感知,智能控制,数据分析。E-mail:xiu.kan@sues.edu.cn

  • 中图分类号: TP391

Image Fusion Based on NSCT and Minimum-Local Mean Gradient

  • 摘要: 针对传统红外图像与可见光图像融合存在对比度低、细节丢失、目标模糊等问题,本文基于非下采样轮廓波变换(Non-subsampled Contourlet Transform,NSCT)的思想,通过改进权重函数和融合规则,建立新的融合算法实现红外图像和可见光图像的有效融合。首先,通过NSCT变换对红外和可见光图像进行多尺度分解得到对应的低频系数和高频系数。然后,采用改进的最小化规则和局部平均梯度规则分别对低频系数和高频系数进行融合处理,得到对应的最优融合系数,并将所得融合系数进行NSCT逆变换得到最终融合图像。最后,使用公共数据集与其他5种算法进行对比实验,并在7个具有实际意义的性能评价指标约束下,验证所设计算法的有效性和鲁棒性。
  • 图  1  NSCT分解图

    Figure  1.  NSCT decomposition figure

    图  2  红外图像和显著性检测图

    Figure  2.  Infrared image and significance detection diagram

    图  3  式(9)函数对应表达

    Figure  3.  The corresponding expression of equation (9) is the function

    图  4  式(10)函数对应表达

    Figure  4.  The corresponding expression of equation (10) is the function

    图  5  改进前后的融合结果对比图

    Figure  5.  Comparison before and after improvement

    图  6  图像融合流程图

    Figure  6.  Schematic diagram of image fusion

    图  7  “UN Camp”图像的融合结果

    Figure  7.  "UN Camp"image fusion results

    图  8  “Kaptein_1123”图像的融合结果

    Figure  8.  "Kaptein_1123"image fusion results

    图  9  “Quad”图像的融合结果

    Figure  9.  "Quad"image fusion results

    表  1  “UN Camp”图像融合结果的客观评价数据

    Table  1.   Objective evaluation data of fusion results of "UN Camp" image

    LPRP CBF CT DTCWT NSCT-M Ours
    IE 5.8736 5.8778 5.9535 6.6971 6.0300 6.7782
    AG 5.0301 5.2612 5.9634 5.4543 6.9987 8.6568
    SD 18.6783 18.5153 19.3430 28.6416 20.2610 33.7394
    SF 10.5760 10.7665 12.6522 10.9804 14.8598 18.0516
    MI 1.7317 1.7287 1.7345 1.8071 1.7482 1.7807
    VIF 0.2834 0.2889 0.3066 0.4386 0.2932 0.5982
    QABF 0.3531 0.3391 0.4069 0.3956 0.3895 0.4116
    下载: 导出CSV

    表  2  “Kaptein_1123”图像的融合结果的客观评价数据

    Table  2.   Objective evaluation data of fusion results of "Kaptein_1123" image

    LPRP CBF CT DTCWT NSCT-M Ours
    IE 6.5108 6.5400 6.4460 6.7483 6.4623 6.7527
    AG 4.7763 6.2218 3.8271 5.1686 4.2784 7.1084
    SD 32.0192 32.6583 30.8624 36.0783 31.3539 47.2670
    SF 11.9578 17.3752 9.9378 15.4644 11.4173 19.4531
    MI 1.6681 1.6794 1.6542 1.6272 1.6570 1.6728
    VIF 0.2852 0.2693 0.2639 0.1811 0.2561 0.4387
    QABF 0.3080 0.2947 0.2552 0.3225 0.2737 0.3362
    下载: 导出CSV

    表  3  “Quad”图像的融合结果客观评价数据

    Table  3.   Objective evaluation data of fusion results of "Quad" image

    LPRP CBF CT DTCWT NSCT-M Ours
    IE 5.7331 5.8807 6.4628 6.0780 5.7632 6.0501
    AG 2.6998 3.1727 2.9816 3.0675 2.9061 3.4874
    SD 29.8808 28.9345 34.2477 39.7308 26.0337 35.0966
    SF 6.6144 7.5910 7.1388 7.4617 7.1121 8.1971
    MI 1.7991 1.7882 1.8732 1.8816 1.7565 1.8579
    VIF 0.1881 0.2749 0.3262 0.2403 0.2637 0.2963
    QABF 0.2010 0.3970 0.3509 0.2436 0.4100 0.4534
    下载: 导出CSV

    表  4  “UN Camp”融合图像的达成度

    Table  4.   Degree of "UN Camp" fusion image

    LPRP CBF CT DTCWT NSCT-M Ours
    IE 0.8665 0.8672 0.8783 0.9880 0.8896 1.0000
    AG 0.5811 0.6078 0.6889 0.6301 0.8085 1.0000
    SD 0.5536 0.5488 0.5733 0.8489 0.6005 1.0000
    SF 0.5859 0.5964 0.7009 0.6083 0.8232 1.0000
    MI 0.9583 0.9566 0.9598 1.0000 0.9674 0.9854
    VIF 0.4738 0.4829 0.5125 0.7332 0.4901 1.0000
    QABF 0.8579 0.8239 0.9886 0.9611 0.9463 1.0000
    TGA 4.8770 4.8835 5.3023 5.7696 5.5256 6.9854
    下载: 导出CSV

    表  5  “Kaptein_1123”融合图像的达成度

    Table  5.   degree of "Kaptein_1123" fusion image

    LPRP CBF CT DTCWT NSCT-M Ours
    IE 0.9642 0.9685 0.9546 0.9993 0.9570 1.0000
    AG 0.6719 0.8753 0.5384 0.7271 0.6019 1.0000
    SD 0.6774 0.6909 0.6529 0.7633 0.6633 1.0000
    SF 0.6147 0.8932 0.5109 0.7950 0.5869 1.0000
    MI 0.9933 1.0000 0.9850 0.9689 0.9867 0.9961
    VIF 0.6501 0.6139 0.6016 0.4128 0.5838 1.0000
    QABF 0.9161 0.8766 0.7591 0.9593 0.8141 1.0000
    TGA 5.4877 5.9183 5.0024 5.6257 5.1937 6.9961
    下载: 导出CSV

    表  6  “Quad”融合图像的达成度

    Table  6.   degree of "Quad" fusion image

    LPRP CBF CT DTCWT NSCT-M Ours
    IE 0.8871 0.9099 1.0000 0.9405 0.8917 0.9361
    AG 0.7742 0.9098 0.8550 0.8796 0.8333 1.0000
    SD 0.7521 0.7283 0.8620 1.0000 0.6553 0.8834
    SF 0.8069 0.9261 0.8709 0.9103 0.8676 1.0000
    MI 0.9562 0.9504 0.9955 1.0000 0.9335 0.9874
    VIF 0.5766 0.8427 1.0000 0.7367 0.8084 0.9083
    QABF 0.4433 0.8756 0.7739 0.5373 0.9043 1.0000
    TGA 5.1964 6.1428 6.3573 6.0044 5.8941 6.7152
    下载: 导出CSV
  • [1] MA J, MA Y, LI C. Infrared and visible image fusion methods and applications: A survey[J]. Information Fusion, 2019, 45: 153-178. doi:  10.1016/j.inffus.2018.02.004
    [2] HUANG Z, CHEN L, ZHANG Y, et al. Robust contact-point detection from pantograph-catenary infrared images by employing horizontal- vertical enhancement operator[J]. Infrared Physics & Technology, 2019, 101: 146-155.
    [3] HUANG Z, FANG H, LI Q, et al. Optical remote sensing image enhancement with weak structure preservation via spatially adaptive gamma correction[J]. Infrared Physics & Technology, 2018, 94: 38-47.
    [4] 肖中杰.基于NSCT红外与可见光图像融合算法优化研究[J].红外技术, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010

    XIAO Zhongjie. Optimization of Infrared and Visible Image Fusion Algorithm based on NSCT[J]. Infrared Technology, 2017, 39(12): 1127-1130. http://hwjs.nvir.cn/article/id/hwjs201712010
    [5] ZHANG B, LU X, PEI H, et al. A fusion algorithm for infrared and visible images based on saliency analysis and non-subsampled Shearlet transform[J]. Infrared Physics & Technology, 2015, 73: 286-297.
    [6] ZHOU Z, WANG B, LI S, et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30: 15-26. doi:  10.1016/j.inffus.2015.11.003
    [7] MA J, YU W, LIANG P, et al. Fusion GAN: A generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. doi:  10.1016/j.inffus.2018.09.004
    [8] ZHANG Y, ZHANG L, BAI X, et al. Infrared and visual image fusion through infrared feature extraction and visual information preservation[J]. Infrared Physics & Technology, 2017, 83: 227-237.
    [9] 肖进胜, 饶天宇, 贾茜, 等.基于图切割的拉普拉斯金字塔图像融合算法[J].光电子·激光, 2014, 25(7): 1416-1424. https://www.cnki.com.cn/Article/CJFDTOTAL-GDZJ201407032.htm

    XIAO Jinsheng, RAO Tianyu, JIA Qian, et al. Image fusion algorithm of Laplacian Pyramid based on graph Cutting[J]. Photoelectron Laser, 2014, 25(7): 1416-1424. https://www.cnki.com.cn/Article/CJFDTOTAL-GDZJ201407032.htm
    [10] 刘斌, 付忠旺.基于四通道不可分提升小波的多聚焦图像融合[J].系统工程与电子技术, 2018, 40(2): 463-471. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD201802032.htm

    LIU Bin, FU Zhongwang. Multi-focus image fusion based on four-channel Indivisible Lifting Wavelet[J]. Systems Engineering and Electronics technology, 2018, 40(2): 463-471. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD201802032.htm
    [11] 薛琴, 范勇, 李绘卓, 等.基于广义邻域结构相似度的红外和彩色可见光图像融合[J].四川大学学报:自然科学版, 2012, 49(3): 570-576. doi:  10.3969/j.issn.0490-6756.2012.03.017

    XUE Qin, FAN Yong, LI Huizhong, et al. Infrared and color visible light image fusion based on generalized neighborhood structure similarity[J]. Journal of Sichuan University: Natural Science, 2012, 49(3): 570-576. doi:  10.3969/j.issn.0490-6756.2012.03.017
    [12] DA Cunha A L, ZHOU J, DO M N. The nonsubsampled contourlet transform: theory, design, and applications[J]. IEEE Transactions on Image Processing, 2006, 15(10): 3089-3101. doi:  10.1109/TIP.2006.877507
    [13] ZHU Z, YIN H, CHAI Y, et al. A novel multi-modality image fusion method based on image decomposition and sparse representation[J]. Information Sciences, 2018, 432: 516-529. doi:  10.1016/j.ins.2017.09.010
    [14] 甄媚, 王书朋.可见光与红外图像自适应加权平均融合方法[J].红外技术, 2019, 41(4): 341-346. http://hwjs.nvir.cn/article/id/hwjs201904008

    ZHEN Mei, WANG Shupeng. Adaptive Weighted average fusion method for Visible light and infrared Images[J]. Infrared Technology, 2019, 41(4): 341-346. http://hwjs.nvir.cn/article/id/hwjs201904008
    [15] XIA Z, PENG X, FENG X, et al. Scarce face recognition via two-layer collaborative representation[J]. IET Biometrics, 2017, 7(1): 56-62.
    [16] 张新曼, 韩九强.基于视觉特性的多尺度对比度塔图像融合及性能评价[J].西安交通大学学报, 2004(4): 380-383. doi:  10.3321/j.issn:0253-987X.2004.04.013

    ZHANG Xinman, HAN Jiuqiang. Image fusion and Performance Evaluation of Multi-scale Contrast tower based on visual Characteristics[J]. Journal of Xi 'an Jiaotong University, 2004(4): 380-383. doi:  10.3321/j.issn:0253-987X.2004.04.013
    [17] 徐丹萍, 王海梅.基于双边滤波和NSST的红外与可见光图像融合[J].计算机测量与控制, 2018, 26(4): 201-204. https://www.cnki.com.cn/Article/CJFDTOTAL-JZCK201804053.htm

    XU Danping, WANG Haimei. Infrared and visible image fusion based on bilateral Filtering and NSST[J]. Computer Measurement and Control, 2018, 26(4): 201-204. https://www.cnki.com.cn/Article/CJFDTOTAL-JZCK201804053.htm
    [18] 李晖晖, 郭雷, 刘坤.基于曲波变换的遥感图像融合研究[J].光电子·激光, 2008(3): 400-403, 411. doi:  10.3321/j.issn:1005-0086.2008.03.029

    LI Huihui, GUO Lei, LIU Kun. Research on Remote sensing Image Fusion based on Qu Wave Transform[J]. Optoelectronics.Laser, 2008(3): 400-403, 411. doi:  10.3321/j.issn:1005-0086.2008.03.029
    [19] 王亚杰, 李殿起, 徐心和.基于双树复小波变换彩色多聚焦图像融合方法[J].计算机工程与应用, 2007(28): 12-14, 40. doi:  10.3321/j.issn:1002-8331.2007.28.005

    WANG Yajie, LI Dianqi, XU Xinhe. Color Multi-focus image fusion method based on Double tree Complex Wavelet Transform[J]. Computer Engineering and Applications, 2007(28): 12-14, 40. doi:  10.3321/j.issn:1002-8331.2007.28.005
    [20] JIN X, JIANG Q, YAO S, et al. A survey of infrared and visual image fusion methods[J]. Infrared Physics & Technology, 2017, 85: 478-501.
    [21] 王跃华, 陶忠祥.红外与可见光图像融合质量评价方法综述[J].红外, 2012, 33(6): 7-11. doi:  10.3969/j.issn.1672-8785.2012.06.002

    WANG Yuehua, TAO Zhongxiang. Review of evaluation methods of infrared and visible image fusion quality[J]. Infrared, 2012, 33(6): 7-11. doi:  10.3969/j.issn.1672-8785.2012.06.002
    [22] QU G, ZHANG D, YAN P. Information measure for performance of image fusion[J]. Electronics letters, 2002, 38(7): 313-315. doi:  10.1049/el:20020212
    [23] HAN Y, CAI Y, CAO Y, et al. A new image fusion performance metric based on visual information fidelity[J]. Information Fusion, 2013, 14(2): 127-135. doi:  10.1016/j.inffus.2011.08.002
    [24] 薄瑞峰, 苗鸿宾.目标达成度评价法在机械设计课程考核中的应用[J].机械设计, 2018, 35(S2): 236-238. https://www.cnki.com.cn/Article/CJFDTOTAL-JXSJ2018S2078.htm

    BO Ruifeng, MIAO Hongbin. Application of objective Achievement Evaluation method in mechanical Design Course Assessment[J]. Mechanical Design, 2018, 35(S2): 236-238. https://www.cnki.com.cn/Article/CJFDTOTAL-JXSJ2018S2078.htm
  • 加载中
图(9) / 表(6)
计量
  • 文章访问数:  266
  • HTML全文浏览量:  76
  • PDF下载量:  33
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-04-02
  • 修回日期:  2021-01-05
  • 刊出日期:  2021-01-20

目录

    /

    返回文章
    返回