Advanced Search
中文
Volume 45 Issue 9
Sep.  2023
Turn off MathJax
Article Contents
Abstract
References
Related Articles
WU Lingxiao, KANG Jiayin, JI Yunxiang. Infrared and Visible Image Fusion Based on Guided Filter and Sparse Representation in NSST Domain[J]. Infrared Technology , 2023, 45(9): 915-924.
Citation: WU Lingxiao, KANG Jiayin, JI Yunxiang. Infrared and Visible Image Fusion Based on Guided Filter and Sparse Representation in NSST Domain[J]. Infrared Technology , 2023, 45(9): 915-924.
shu

Infrared and Visible Image Fusion Based on Guided Filter and Sparse Representation in NSST Domain

  • School of Electronic Engineering, Jiangsu Ocean University, Lianyungang 222005, China

More Information
  • Received Date: August 01, 2022
  • Revised Date: September 12, 2022
    Graphical Abstract
    • Abstract
  • Image fusion technology aims to solve the problem of insufficient and incomplete information provided by a single-modality image. This paper proposes a novel method based on guided filter (GF) and sparse representation (SR) in the non-subsampled shearlet transform (NSST) domain, to fuse infrared and visible images. Specifically, ① the infrared and visible images are respectively decomposed using NSST to obtain the corresponding high-frequency and low-frequency sub-band images; ② The GF-weighted fusion strategy is exploited to fuse the high-frequency sub-band images; ③ Rolling guidance filter (RGF) is used to further decompose the low-frequency sub-band images into base and detail layers, whereby the base layers are fused via SR, and the detail layers are fused using local maximum strategy which is based on consistency verification; ④ An inverse NSST is performed on the fused high-frequency and low-frequency sub-band images to obtain the final fusion result. Compared to those of other methods, experimental results on public datasets show that the fusion result obtained by the proposed method has richer texture detail and better subjective visual effects. In addition, the proposed method achieves overall better performance in terms of objective metrics that are commonly used for evaluating fusion results.
    • FullText(HTML)
    • References (30)
  • [1]
    MA J, MA Y, LI C. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
    [2]
    LIU Y P, JIN J, WANG Q, et al. Region level based multi-focus image fusion using quaternion wavelet and normalized cut[J]. Signal Processing, 2014, 97: 9-30. DOI: 10.1016/j.sigpro.2013.10.010
    [3]
    Toet A. Image fusion by a ratio of low-pass pyramid[J]. Pattern Recognition Letters, 1989, 9(4): 245-253. DOI: 10.1016/0167-8655(89)90003-2
    [4]
    Choi M, Kim R Y, Nam M R, et al. Fusion of multispectral and panchromatic satellite images using the curvelet transform[J]. IEEE Geoscience and Remote Sensing Letters, 2005, 2(2): 136-140. DOI: 10.1109/LGRS.2005.845313
    [5]
    Easley G, Labate D, Lim W Q. Sparse directional image representations using the discrete shearlet transform[J]. Applied and Computational Harmonic Analysis, 2008, 25(1): 25-46. DOI: 10.1016/j.acha.2007.09.003
    [6]
    康家银, 陆武, 张文娟. 融合NSST和稀疏表示的PET和MRI图像融合[J]. 小型微型计算机系统, 2019, 40(12): 2506-2511. https://www.cnki.com.cn/Article/CJFDTOTAL-XXWX201912006.htm

    KANG J Y, LU W, ZHANG W J. Fusion of PET and MRI images using non-subsampled shearlet transform combined with sparse representation[J]. Journal of Chinese Computer Systems. 2019, 40(12): 2506-2511. https://www.cnki.com.cn/Article/CJFDTOTAL-XXWX201912006.htm
    [7]
    LIU Z W, FENG Y, CHEN H, et al. A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain[J]. Optics and Lasers in Engineering, 2017, 97: 71-77. DOI: 10.1016/j.optlaseng.2017.05.007
    [8]
    董安勇, 杜庆治, 苏斌, 等. 基于卷积神经网络的红外与可见光图像融合[J]. 红外技术, 2020, 42(7): 660-669. http://hwjs.nvir.cn/article/id/hwjs202007009

    DONG A Y, DU Q Z, SU B, et al. Infrared and visible image fusion based on convolutional neural network[J]. Infrared Technology, 2020, 42(7): 660-669. http://hwjs.nvir.cn/article/id/hwjs202007009
    [9]
    叶坤涛, 李文, 舒蕾蕾, 等. 结合改进显著性检测与NSST的红外与可见光图像融合方法[J]. 红外技术, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805

    YE K T, LI W, SHU L L, et al. Infrared and visible image fusion method based on improved saliency detection and non-subsampled shearlet transform[J]. Infrared Technology, 2021, 43(12): 1212-1221. http://hwjs.nvir.cn/article/id/bfd9f932-e0bd-4669-b698-b02d42e31805
    [10]
    王晓娜, 潘晴, 田妮莉. 基于NSST-DWT-ICSAPCNN的多模态图像融合算法[J]. 红外技术, 2022, 44(5): 497-503. http://hwjs.nvir.cn/article/id/0644931d-58ad-4bbd-a752-5f4bbd2061e1

    WANG X N, PAN Q, TIAN N L. Multi-modality image fusion algorithm based on NSST-DWT-ICSAPCNN[J]. Infrared Technology, 2022, 44(5): 497-503. http://hwjs.nvir.cn/article/id/0644931d-58ad-4bbd-a752-5f4bbd2061e1
    [11]
    常莉红. 基于剪切波变换和稀疏表示理论的图像融合方法[J]. 中山大学学报: 自然科学版, 2017, 56(4): 16-19. https://www.cnki.com.cn/Article/CJFDTOTAL-ZSDZ201704003.htm

    CHANG L H. Fusion method based on shearlet transform and sparse representation[J]. Acta Scientiarum Naturalium Universitatis Sunyatsen, 2017, 56(4): 16-19. https://www.cnki.com.cn/Article/CJFDTOTAL-ZSDZ201704003.htm
    [12]
    王相海, 邢俊宇, 王鑫莹, 等. 基于剪切波和低秩稀疏表示的噪声图像融合算法研究[J]. 辽宁师范大学学报: 自然科学版, 2022, 45(2): 191-200. https://www.cnki.com.cn/Article/CJFDTOTAL-LNSZ202202008.htm

    WANG X H, XING J Y, WANG X Y, et al. Noisy image fusion algorithm based on shearlet and low-rank sparse representation[J]. Journal of Liaoning Normal University (Natural Science Edition), 2022, 45(2): 191-200. https://www.cnki.com.cn/Article/CJFDTOTAL-LNSZ202202008.htm
    [13]
    吴月. 基于非下采样剪切波变换和稀疏表示的图像融合算法研究[D]. 北京: 北京交通大学, 2018.

    WU Y. Image Fusion Algorithm Based on Sparse Representation and Non-Subsampled Shearlet Transform[D]. Beijing: Beijing Jiaotong University, 2018.
    [14]
    HE K M, SUN J, TANG X O. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409. DOI: 10.1109/TPAMI.2012.213
    [15]
    ZHANG Q, SHEN X, XU L, et al. Rolling guidance filter[C]//13th European Conference on Computer Vision, 2014: 815-830.
    [16]
    MA J L, ZHOU Z Q, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.
    [17]
    Aharon M, Elad M, Bruckstein A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322. DOI: 10.1109/TSP.2006.881199
    [18]
    YANG B, LI S T. Multifocus image fusion and restoration with sparse representation[J]. IEEE Transactions on Instrumentation and Measurement, 2010, 59(4): 884-892. DOI: 10.1109/TIM.2009.2026612
    [19]
    LI H, Manjunath B S, Mitra S K. Multisensor image fusion using the wavelet transform[J]. Graphical Models and Image Processing, 1995, 57(3): 235-245.
    [20]
    MA J Y, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109.
    [21]
    LIU Y, WANG Z F. Simultaneous image fusion and denoising with adaptive sparse representation[J]. IET Image Processing, 2015, 9(5): 347-357.
    [22]
    LI S T, KANG X D, HU J W. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875.
    [23]
    LIU Y, LIU S P, WANG Z F. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164.
    [24]
    LIU Y, CHEN X, Ward R K, et al. Image fusion with convolutional sparse representation[J]. IEEE Signal Processing Letters, 2016, 23(12): 1882-1886.
    [25]
    SHEN Y, NA J, WU Z D, et al. Tetrolet transform images fusion algorithm based on fuzzy operator[J]. Journal of Frontiers of Computer Science and Technology, 2015, 9(9): 1132.
    [26]
    敬忠良, 肖刚, 李振华. 图像融合—理论与应用[M]. 北京: 高等教育出版社, 2007.

    JING Z L, XIAO G, LI Z H. Image Fusion: Theory and Applications[M]. Beijing: High Education Press, 2007. (in Chinese)
    [27]
    ZHENG Y, Essock E A, Hansen B C, et al. A new metric based on extended spatial frequency and its application to DWT based fusion algorithms[J]. Information Fusion, 2007, 8(2): 177-192.
    [28]
    Xydeas C S, Petrovic V S. Objective pixel-level image fusion performance measure[C]//AeroSense, 2000: 89-98.
    [29]
    WANG Q, SHEN Y, JIN J. Performance Evaluation of Image Fusion Techniques[M]. Amsterdam: Elsevier, 2008: 469-492.
    [30]
    Piella G, Heijmans H. A new quality metric for image fusion[C]// International Conference on Image Processing, IEEE, 2003(2): Ⅲ-173-6.
    • Related Articles

  • Related Articles

    [1]TIAN Yafang, SHI Yanli, LI Fangjiang. Research Progress of InAs/GaSb Type-Ⅱ Superlattice Long-wave Infrared Detector[J]. Infrared Technology , 2023, 45(8): 799-807.
    [2]QU Fan, FU Jianyu, HOU Ying, LU Yihong, LI Zhenfeng, CHEN Dapeng. Optimizing Diode Structure in Uncooled Infrared Focal Plane Array[J]. Infrared Technology , 2023, 45(3): 308-314.
    [3]LIU Jiwei, WANG Jinhua, SUN Junwei, HU Hanlin, CHEN Wenli. Ceramic Package Structure Optimization and Reliability Analysis for Uncooled Infrared Detectors[J]. Infrared Technology , 2023, 45(1): 77-83.
    [4]ZHANG Jiangfeng, DIAO Yunfei, ZHANG Xiaoling, MENG Qingduan. Thermal Stress Structural Optimization of InSb Infrared Focal Plane Array Detector[J]. Infrared Technology , 2021, 43(12): 1202-1206.
    [5]XIAO Rishi, CHEN Xiaoping. Experimental Study on Miniature Joule-Thomson Cooler for Analyzing Steady-state Characteristics with Different Parameters and Operating Conditions[J]. Infrared Technology , 2018, 40(8): 818-823.
    [6]XIONG Chao, XIA Ming, CHEN Jun, HUANG Wei, XI Zhongli, ZOU Dingli, SU Jun, HUAN Jian, LIU Yan. Numerical Simulation and Experiment Analysis of Phasor Shift of a Micro Pulse Tube Cryocooler[J]. Infrared Technology , 2017, 39(6): 558-563.
    [7]Numerical Simulation of Lock-in Thermograpy for Infrared Nondestructive Testing[J]. Infrared Technology , 2013, (2): 119-122.
    [8]Numerical Analysis on Flow and Heat Transfer Characteristics of Rectangular Micro-channel[J]. Infrared Technology , 2010, 32(5): 304-308. DOI: 10.3969/j.issn.1001-8891.2010.05.012
    [9]MA Yan-bing, LIU Tao, ZOU Peng-cheng, JIANG Jun, GAO Wen-feng, LUO Chuan-xu, LIN Wen-xian. Numerical Simulation and Analysis of CZT Crystal Growth by the VB Method[J]. Infrared Technology , 2009, 31(4): 240-245. DOI: 10.3969/j.issn.1001-8891.2009.04.014
    [10]CHEN Xiao-Ping, XIA Ming. Numerical Analysis of the Integration Stirling-Cycle Refrigerator Applied to Focal Plane Array Detector[J]. Infrared Technology , 2006, 28(1): 54-58. DOI: 10.3969/j.issn.1001-8891.2006.01.014

  • Cited by

    Periodical cited type(0)

    Other cited types(2)

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by(2)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return