Advanced Search
中文
Volume 44 Issue 5
May  2022
Turn off MathJax
Article Contents
Abstract
References
Related Articles
WANG Xiaona, PAN Qing, TIAN Nili. Multi-modality Image Fusion Algorithm Based on NSST-DWT-ICSAPCNN[J]. Infrared Technology , 2022, 44(5): 497-503.
Citation: WANG Xiaona, PAN Qing, TIAN Nili. Multi-modality Image Fusion Algorithm Based on NSST-DWT-ICSAPCNN[J]. Infrared Technology , 2022, 44(5): 497-503.
shu

Multi-modality Image Fusion Algorithm Based on NSST-DWT-ICSAPCNN

  • Faculty of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China

More Information
  • Received Date: September 01, 2021
  • Revised Date: November 23, 2021
    Graphical Abstract
    • Abstract
  • To increase the information of the fused image, this paper proposes an improved multi-modality image fusion algorithm that combines the complementary advantages of the non-subsampled shearlet transform (NSST) and discrete wavelet transform (DWT). NSST was used to decompose the two source images in multiscale and multi-direction to obtain the corresponding high-frequency and low-frequency sub-bands. The low-frequency sub-bands were further decomposed into low-frequency energy sub-bands and low-frequency detail sub-bands by the DWT, and the low-frequency energy sub-bands were fused by the maximum selection rules. An adaptive pulse-coupled neural network with improved connection strength (ICSAPCNN) was used to fuse the detailed sub-bands and high-frequency sub-bands, and the energy sub-bands and detailed sub-bands were fused by inverse DWT to obtain the fused low-frequency sub-bands. The NSST inverse transform was used to reconstruct the fusion image with rich details. The experimental results verified that the proposed algorithm is superior to the other algorithms in both subjective vision and objective evaluation and can be applied to the fusion of both infrared and visible source images and medical source images.
    • FullText(HTML)
    • References (18)
  • [1]
    YANG Y, QUE Y, HUANG S, et al. Multimodal sensor medical image fusion based on type-2 fuzzy logic in NSCT domain[J]. IEEE Sensors Journal, 2016, 16(10): 3735-3745. DOI: 10.1109/JSEN.2016.2533864
    [2]
    LI G, LIN Y, QU X. An infrared and visible image fusion method based on multi-scale transformation and norm optimization[J]. Information Fusion, 2021, 71: 109-129. DOI: 10.1016/j.inffus.2021.02.008
    [3]
    LI X, ZHOU F, TAN H. Joint image fusion and denoising via three-layer decomposition and sparse representation[J]. Knowledge-Based Systems, 2021, 224: 107087. DOI: 10.1016/j.knosys.2021.107087
    [4]
    XU H, MA J. EMFusion: An unsupervised enhanced medical image fusion network[J]. Information Fusion, 2021, 76: 177-186. . DOI: 10.1016/j.inffus.2021.06.001
    [5]
    Bulanon D M, Burks T F, Alchanatis V. Image fusion of visible and thermal images for fruit detection[J]. Biosystems Engineering, 2009, 103(1): 12-22. DOI: 10.1016/j.biosystemseng.2009.02.009
    [6]
    ZHAN L, ZHUANG Y, HUANG L. Infrared and visible images fusion method based on discrete wavelet transform[J]. J. Comput. , 2017, 28(2): 57-71.
    [7]
    LIU Y, WANG Z. Simultaneous image fusion and denoising with adaptive sparse representation[J]. IET Image Processing, 2015, 9(5): 347-357. DOI: 10.1049/iet-ipr.2014.0311
    [8]
    LIU Y, CHEN X, CHENG J, et al. Infrared and visible image fusion with convolutional neural networks[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16(3): 1850018. DOI: 10.1142/S0219691318500182
    [9]
    ZHU Z, ZHENG M, QI G, et al. A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain[J]. IEEE Access, 2019, 7: 20811-20824. DOI: 10.1109/ACCESS.2019.2898111
    [10]
    ZHANG L, ZENG G, WEI J, et al. Multi-modality image fusion in adaptive-parameters SPCNN based on inherent characteristics of image[J]. IEEE Sensors Journal, 2019, 20(20): 11820-11827.
    [11]
    张蕾. 采用改进平均梯度与自适应PCNN的图像融合[J]. 计算机应用与软件, 2021, 38(3): 218-223. DOI: 10.3969/j.issn.1000-386x.2021.03.033

    ZHANG Lei. Image fusion using improved average gradient and adaptive PCNN[J]. Computer Application and Software, 2021, 38(3): 218-223. DOI: 10.3969/j.issn.1000-386x.2021.03.033
    [12]
    YIN M, LIU X, LIU Y, et al. Medical image fusion with parameter- adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2018, 68(1): 49-64.
    [13]
    Diwakar M, Singh P, Shankar A. Multi-modal medical image fusion framework using co-occurrence filter and local extrema in NSST domain[J]. Biomedical Signal Processing and Control, 2021, 68: 102788. DOI: 10.1016/j.bspc.2021.102788
    [14]
    邓辉, 王长龙, 胡永江, 等. 脉冲耦合神经网络在图像融合中的应用研究[J]. 电光与控制, 2019, 26(11): 19-24. https://www.cnki.com.cn/Article/CJFDTOTAL-DGKQ201911006.htm

    DENG Hui, WANG Changlong, HU Yongjiang, et al. Application of pulse coupled neural network in image fusion[J]. Electronics Options & Contral, 2019, 26(11): 19-24. https://www.cnki.com.cn/Article/CJFDTOTAL-DGKQ201911006.htm
    [15]
    杨风暴, 董安冉, 张雷, 等. DWT、NSCT和改进PCA协同组合红外偏振图像融合[J]. 红外技术, 2017, 39(3): 201-208. http://hwjs.nvir.cn/article/id/hwjs201703001

    YANG Fengbao, DONG Aran, ZHANG Lei, et al. Infrared Polarization Image fusion using the synergistic combination of DWT, NSCT and improved PCA[J]. Infrared Technology, 2017, 39(3): 201-208. http://hwjs.nvir.cn/article/id/hwjs201703001
    [16]
    TAN W, Tiwari P, Pandey H M, et al. Multimodal medical image fusion algorithm in the era of big data[J]. Neural Computing and Applications, 2020: 1-21.
    [17]
    JIANG L, ZHANG D, CHE L. Texture analysis-based multi-focus image fusion using a modified Pulse-Coupled Neural Network (PCNN)[J]. Signal Processing: Image Communication, 2021, 91: 116068. DOI: 10.1016/j.image.2020.116068
    [18]
    LIU Z, Blasch E, XUE Z, et al. Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(1): 94-109.
    • Related Articles

  • Related Articles

    [1]LIAO Guangfeng, GUAN Zhiwei, CHEN Qiang. An Improved Dual Discriminator Generative Adversarial Network Algorithm for Infrared and Visible Image Fusion[J]. Infrared Technology , 2025, 47(3): 367-375.
    [2]YUAN Hongchun, ZHANG Bo, CHENG Xin. Underwater Image Enhancement Algorithm Combining Transformer and Generative Adversarial Network[J]. Infrared Technology , 2024, 46(9): 975-983.
    [3]LI Li, YI Shi, LIU Xi, CHENG Xinghao, WANG Cheng. Infrared Image Deblurring Based on Dense Residual Generation Adversarial Network[J]. Infrared Technology , 2024, 46(6): 663-671.
    [4]DI Jing, REN Li, LIU Jizhao, GUO Wenqing, LIAN Jing. Infrared and Visible Image Fusion Based on Three-branch Adversarial Learning and Compensation Attention Mechanism[J]. Infrared Technology , 2024, 46(5): 510-521.
    [5]CHEN Xin. Infrared and Visible Image Fusion Using Double Attention Generative Adversarial Networks[J]. Infrared Technology , 2023, 45(6): 639-648.
    [6]WANG Tianyuan, LUO Xiaoqing, ZHANG Zhancheng. Infrared and Visible Image Fusion Based on Self-attention Learning[J]. Infrared Technology , 2023, 45(2): 171-177.
    [7]FU Tian, DENG Changzheng, HAN Xinyue, GONG Mengqing. Infrared and Visible Image Registration for Power Equipments Based on Deep Learning[J]. Infrared Technology , 2022, 44(9): 936-943.
    [8]LI Yunhong, LIU Yudong, SU Xueping, LUO Xuemin, YAO Lan. Review of Infrared and Visible Image Registration[J]. Infrared Technology , 2022, 44(7): 641-651.
    [9]HUANG Mengtao, GAO Na, LIU Bao. Image Deblurring Method Based on a Dual-Discriminator Weighted Generative Adversarial Network[J]. Infrared Technology , 2022, 44(1): 41-46.
    [10]LUO Di, WANG Congqing, ZHOU Yongjun. A Visible and Infrared Image Fusion Method based on Generative Adversarial Networks and Attention Mechanism[J]. Infrared Technology , 2021, 43(6): 566-574.

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by()
    Related

    Copyright © 《Infrared Technology》Editorial Office京ICP备000000000号-1

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return