Advanced Search
中文
Volume 43 Issue 6
Jun.  2021
Turn off MathJax
Article Contents
Abstract
References
Related Articles
LUO Di, WANG Congqing, ZHOU Yongjun. A Visible and Infrared Image Fusion Method based on Generative Adversarial Networks and Attention Mechanism[J]. Infrared Technology , 2021, 43(6): 566-574.
Citation: LUO Di, WANG Congqing, ZHOU Yongjun. A Visible and Infrared Image Fusion Method based on Generative Adversarial Networks and Attention Mechanism[J]. Infrared Technology , 2021, 43(6): 566-574.
shu

A Visible and Infrared Image Fusion Method based on Generative Adversarial Networks and Attention Mechanism

  • 1.

    College of Automation Engineering of Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

  • 2.

    Science and Technology on Near-Surface Detection Laboratory, Wuxi 214000, China

More Information
  • Received Date: September 07, 2020
  • Revised Date: October 11, 2020
    Graphical Abstract
    • Abstract
  • A new fusion method for visible and infrared images based on generative adversarial networks is proposed to solve the problem of recognizing targets in low-light images; the method can be directly applied to the fusion of RGB three-channel visible images and infrared images. In generative adversarial networks, the generator adopts a U-Net structure with encoding and decoding layers. The discriminator adopts a Markovian discriminator, and the attention mechanism is introduced to force the fused image to pay more attention to the high-intensity information on infrared images. The experimental results show that the proposed method not only maintains the detailed texture information of visible images but also introduces the main target information of infrared images to generate fusion images with good visual effects and high target identification, and it performs well in information entropy, structural similarity, and other objective indexes.
    • FullText(HTML)
    • References (16)
  • [1]
    MA J, MA Y, LI C. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
    [2]
    Burt P J, Adelson E H. The Laplacian pyramid as a compact image code[J]. Readings in Computer Vision, 1987, 31(4): 671-679. https://www.sciencedirect.com/science/article/pii/B9780080515816500659
    [3]
    Selesnick I W, Baraniuk R G, Kingsbury N C. The dual-tree complex wavelet transform[J]. IEEE Signal Processing Magazine, 2005, 22(6): 123-151. DOI: 10.1109/MSP.2005.1550194
    [4]
    A L da Cunha, J Zhou, M N Do. Nonsubsampled contourilet transform: filter design and applications in denoising[C]//IEEE International Conference on Image Processing 2005, 749: (doi: 10.1109/ICIP.2005.1529859).
    [5]
    Hariharan H, Koschan A, Abidi M. The direct use of curvelets in multifocus fusion[C]//16th IEEE International Conference on Image Processing (ICIP), 2009: 2185-2188(doi: 10.1109/ICIP.2009.5413840).
    [6]
    LI Hui. Dense fuse: a fusion approach to infrared and visible images[C]//IEEE Transactions on Image Processing, 2018, 28: 2614- 2623(doi: 0.1109/TIP.2018.2887342).
    [7]
    MA J, YU W, LIANG P, et al. Fusion GAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
    [8]
    Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-assisted Intervention, 2015: 234-241.
    [9]
    Hwang S, Park J, Kim N, et al. Multispectral pedestrian detection: Benchmark dataset and baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1037-1045.
    [10]
    Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014: 2672-2680.
    [11]
    Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[J/OL][2015-11-07]. arXiv preprint arXiv: 1511.06434, 2015: https://arxiv.org/abs/1511.06434v1.
    [12]
    MAO X, LI Q, XIE H, et al. Least squares generative adversarial networks[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017: 2813-2821(doi: 10.1109/ICCV.2017.304).
    [13]
    Isola Phillip, ZHU Junyan, ZHOU Tinghui, et al. Image-to-image translation with conditional adversarial networks, 2017: 5967-5976 (doi: 10.1109/CVPR.2017.632).
    [14]
    Jaderberg M, Simonyan K, Zisserman A. Spatial transformer networks[C]//Advances in Neural Information Processing Systems, 2015: 2017-2025.
    [15]
    HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
    [16]
    Woo S, Park J, Lee J Y, et al. Cbam: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.
    • Related Articles

  • Related Articles

    [1]LIAO Guangfeng, GUAN Zhiwei, CHEN Qiang. An Improved Dual Discriminator Generative Adversarial Network Algorithm for Infrared and Visible Image Fusion[J]. Infrared Technology , 2025, 47(3): 367-375.
    [2]YUAN Hongchun, ZHANG Bo, CHENG Xin. Underwater Image Enhancement Algorithm Combining Transformer and Generative Adversarial Network[J]. Infrared Technology , 2024, 46(9): 975-983.
    [3]LI Li, YI Shi, LIU Xi, CHENG Xinghao, WANG Cheng. Infrared Image Deblurring Based on Dense Residual Generation Adversarial Network[J]. Infrared Technology , 2024, 46(6): 663-671.
    [4]DI Jing, REN Li, LIU Jizhao, GUO Wenqing, LIAN Jing. Infrared and Visible Image Fusion Based on Three-branch Adversarial Learning and Compensation Attention Mechanism[J]. Infrared Technology , 2024, 46(5): 510-521.
    [5]CHEN Xin. Infrared and Visible Image Fusion Using Double Attention Generative Adversarial Networks[J]. Infrared Technology , 2023, 45(6): 639-648.
    [6]WANG Tianyuan, LUO Xiaoqing, ZHANG Zhancheng. Infrared and Visible Image Fusion Based on Self-attention Learning[J]. Infrared Technology , 2023, 45(2): 171-177.
    [7]FU Tian, DENG Changzheng, HAN Xinyue, GONG Mengqing. Infrared and Visible Image Registration for Power Equipments Based on Deep Learning[J]. Infrared Technology , 2022, 44(9): 936-943.
    [8]LI Yunhong, LIU Yudong, SU Xueping, LUO Xuemin, YAO Lan. Review of Infrared and Visible Image Registration[J]. Infrared Technology , 2022, 44(7): 641-651.
    [9]HUANG Mengtao, GAO Na, LIU Bao. Image Deblurring Method Based on a Dual-Discriminator Weighted Generative Adversarial Network[J]. Infrared Technology , 2022, 44(1): 41-46.
    [10]LUO Di, WANG Congqing, ZHOU Yongjun. A Visible and Infrared Image Fusion Method based on Generative Adversarial Networks and Attention Mechanism[J]. Infrared Technology , 2021, 43(6): 566-574.

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by()
    Related

    Copyright © 《Infrared Technology》Editorial Office京ICP备000000000号-1

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return