TIAN Lifan, YANG Shen, LIANG Jiaming, WU Jin. Infrared and Visible Image Fusion Based on SGWT and Multi-Saliency[J]. Infrared Technology , 2022, 44(7): 676-685.
Citation: TIAN Lifan, YANG Shen, LIANG Jiaming, WU Jin. Infrared and Visible Image Fusion Based on SGWT and Multi-Saliency[J]. Infrared Technology , 2022, 44(7): 676-685.

Infrared and Visible Image Fusion Based on SGWT and Multi-Saliency

More Information
  • Received Date: July 31, 2021
  • Revised Date: October 20, 2021
  • Spectral graph wavelet transform (SGWT) can fully utilize the spectral characteristics of an image in the image domain and has advantages in the expression of small irregular regions. Therefore, this paper proposes an infrared and visible fusion algorithm based on multi saliency. First, SGWT is used to decompose the source image into a low-frequency sub-band and several high-frequency sub-bands. For low-frequency coefficients, a multi saliency fusion rule suitable for human visual features is proposed by combining multiple complementary low-level features. For high-frequency coefficients, a rule for increasing the absolute value of the region is proposed by fully considering the correlation of neighborhood pixels. Finally, a weighted least squares optimization method is applied to optimize the fusion image reconstructed by spectral wavelet reconstruction, which highlights the main target and retains the background details of visible light as much as possible. The experimental results show that, compared with seven related algorithms such as DWT and NSCT, this method can highlight the infrared target and retain more visible background details, resulting in a better visual effect. Moreover, it exhibits advantages in four objective evaluations: variance, entropy, Qabf, and mutual information.
  • [1]
    Goshtasby A, Nikolov S. Image fusion: Advances in the state of the art[J]. Information Fusion, 2007, 8(2): 114-118. DOI: 10.1016/j.inffus.2006.04.001
    [2]
    Toet A, Hogervorst M A, Nikolov S G, et al. Towards cognitive image fusion[J]. Information Fusion, 2010, 11(2): 95-113. DOI: 10.1016/j.inffus.2009.06.008
    [3]
    Falk H. Prolog to a categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application[J]. Proceedings of the IEEE, 1999, 87(8): 1315-1326 DOI: 10.1109/5.775414
    [4]
    GAO Y, MA J, Yuille A L. Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2545-2560. DOI: 10.1109/TIP.2017.2675341
    [5]
    LIU C H, QI Y, DING W R. Infrared and visible image fusion method based on saliency detection in sparse domain[J]. Infrared Physics & Technology, 2017, 83: 94-102.
    [6]
    杨风暴, 董安冉, 张雷, 等. DWT、NSCT和改进PCA协同组合红外偏振图像融合[J]. 红外技术, 2017, 39(3): 201-208. http://hwjs.nvir.cn/article/id/hwjs201703001

    YANG Fengbao, DONG Anran, ZHANG Lei, et al. Infrared polarization image fusion using the synergistic combination of DWT, NSCT and improved PCA[J]. Infrared Technology, 2017, 39(3): 201-208. http://hwjs.nvir.cn/article/id/hwjs201703001
    [7]
    董安勇, 杜庆治, 苏斌, 等. 基于卷积神经网络的红外与可见光图像融合[J]. 红外技术, 2020, 42(7): 660-669. http://hwjs.nvir.cn/article/id/hwjs202007009

    DONG Anyong, DU Qingzhi, SU Bin, ZHAO Wenbo, et al. Infrared and visible image fusion based on convolutional neural network[J]. Infrared Technology, 2020, 42(7): 660-669. http://hwjs.nvir.cn/article/id/hwjs202007009
    [8]
    MA J, MA Y, LI C. Infrared and visible image fusion methods and applications: A survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
    [9]
    David K Hammond, Pierre Vandergheynst, Rémi Gribonval. Wavelets on graphs via spectral graph theory[J]. Applied and Computational Harmonic Analysis, 2011, 30: 129-150. DOI: 10.1016/j.acha.2010.04.005
    [10]
    Morrone M C, Ross J, Burr D C, et al. Mach bands are phase dependent[J]. Nature, 1986, 324(6094): 250-253. DOI: 10.1038/324250a0
    [11]
    ZHANG L, ZHANG L, MOU X, et al. FSIM: A feature similarity index for image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(8): 2378-2386. DOI: 10.1109/TIP.2011.2109730
    [12]
    ZHOU Z, LI S, WANG B. Multi-scale weighted gradient-based fusion for multi-focus images[J]. Information Fusion, 2014, 20: 60-72. DOI: 10.1016/j.inffus.2013.11.005
    [13]
    MA J, ZHOU Z, WANG B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics & Technology, 2017, 82: 8-17.
    [14]
    LI H, Manjunath B S, Mitra S. Multisensor image fusion using the wavelet transform[J]. Graphical Models and Image Processing, 1995, 57(3): 235-245. DOI: 10.1006/gmip.1995.1022
    [15]
    Nencini F, Garzelli A, Baronti S, et al. Remote sensing image fusion using the curvelet transform[J]. Information Fusion, 2007, 8(2): 143-156. DOI: 10.1016/j.inffus.2006.02.001
    [16]
    LI S, KANG X, HU J. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2864-2875. DOI: 10.1109/TIP.2013.2244222
    [17]
    LI S, YANG B, HU J. Performance comparison of different multi-resolution transforms for image fusion[J]. Information Fusion, 2011, 12(2): 74-84. DOI: 10.1016/j.inffus.2010.03.002
    [18]
    YU L, LIU S, WANG Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164. DOI: 10.1016/j.inffus.2014.09.004
    [19]
    ZHU Z, ZHENG M, QI G, et al. A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain[J]. IEEE Access, 2019, 7: 20811-20824. DOI: 10.1109/ACCESS.2019.2898111
    [20]
    张小利, 李雄飞, 李军. 融合图像质量评价指标的相关性分析及性能评估[J]. 自动化学报, 2014, 40(2): 306-315. DOI: 10.3724/SP.J.1004.2014.00306

    ZHANG Xiao-Li, LI Xiong-Fei, LI Jun. Validation and correlation analysis of metrics for evaluating performance of image fusion[J]. Acta Automatica Sinica, 2014, 40(2): 306-315. Doi: 10.3724/SP.J.1004.2014.00306

Catalog

    Article views (163) PDF downloads (40) Cited by()
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return