Advanced Search
中文
Volume 45 Issue 2
Feb.  2023
Turn off MathJax
Article Contents
Abstract
References
Related Articles
WANG Tianyuan, LUO Xiaoqing, ZHANG Zhancheng. Infrared and Visible Image Fusion Based on Self-attention Learning[J]. Infrared Technology , 2023, 45(2): 171-177.
Citation: WANG Tianyuan, LUO Xiaoqing, ZHANG Zhancheng. Infrared and Visible Image Fusion Based on Self-attention Learning[J]. Infrared Technology , 2023, 45(2): 171-177.
shu

Infrared and Visible Image Fusion Based on Self-attention Learning

  • 1.

    School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China

  • 2.

    School of Electronic & Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, China

More Information
  • Received Date: March 05, 2021
  • Revised Date: August 21, 2021
    Graphical Abstract
    • Abstract
  • Due to the lack of image saliency preserving in the existing fusion rules, a self-attention-guided infrared and visible light image fusion method is proposed. First, the feature map and self-attention map of the source images are learnt by the self-attention learning mechanism in the feature learning layer. Next, the self-attention map which can capture the long-distance dependent characteristics of the image is used to design average weighted fusion strategy. Finally, the fused feature maps are reconstructed to obtain the fused image, and the learning of image feature coding, self-attention mechanism, fusion rule, and fused feature decoding are realized by generative adversarial network. Experiments on TNO real-world data show that the learned self-attention unit can represent the salient region and benefit the fusion rule design, the proposed algorithm is better than SOAT infrared and visible image fusion algorithms in objective and subjective evaluation, and it retains the detailed information of visible images and infrared target information of infrared images.
    • FullText(HTML)
    • References (20)
  • [1]
    MA J Y, MA Y, LI C. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
    [2]
    YU X C, GAO G Y, XU J D, et al. Remote sensing image fusion based on sparse representation[C]//2014 IEEE Geoscience and Remote Sensing Symposium, 2014: 2858-2861.
    [3]
    ZHAO W D, LU H C. Medical image fusion and denoising with alternating sequential filter and adaptive fractional order total variation[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(9): 2283-2294. DOI: 10.1109/TIM.2017.2700198
    [4]
    LI Y S, TAO C, et al. Unsupervised multilayer feature learning for satellite image scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(2): 157-161. DOI: 10.1109/LGRS.2015.2503142
    [5]
    JIN X, JIANG Q, et al. A survey of infrared and visual image fusion methods[J]. Information Fusion, 2017, 85: 478-501.
    [6]
    BAI X, ZHANG Y, ZHOU F, et al. Quadtree-based multi-focus image fusion using a weighted focus-measure[J]. Information Fusion, 2015, 22: 105-118. DOI: 10.1016/j.inffus.2014.05.003
    [7]
    BAI X Z. Infrared and visual image fusion through feature extraction by morphological sequential toggle operator[J]. Information Fusion, 2015, 71: 77-86.
    [8]
    LIU Y, CHEN X, PENG H, et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 2017, 36: 191-207. DOI: 10.1016/j.inffus.2016.12.001
    [9]
    MA J Y, YU W, LIANG P W, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
    [10]
    LI H, WU X J. DenseFuse: a fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2019, 28(5): 2614-2623. DOI: 10.1109/TIP.2018.2887342
    [11]
    WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Computer Vision and Pattern Recognition, 2018: 7794-7803.
    [12]
    ZHANG H, GOODFELLOW I, Metaxas D, et al. Self-attention generative adversarial networks[C]//International Conference on Machine Learning, 2020: 7354-7363.
    [13]
    杨晓莉, 蔺素珍. 一种注意力机制的多波段图像特征级融合方法[J]. 西安电子科技大学学报, 2020, 47(1): 123-130. https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD202001018.htm

    YANG X L, LIN S Z. Method for multi-band image feature-level fusion based on attention mechanism[J]. Journal of Xidian University, 2020, 47(1): 123-130. https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD202001018.htm
    [14]
    JIAN L, YANG X, LIU Z, et al. A symmetric encoder-decoder with residual block for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-15.
    [15]
    LOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning, 2015, 37: 448-456.
    [16]
    ZHANG Y, LIU Y, SUN P, et al. IFCNN: a general image fusion framework based on convolutional neural network [J]. Information Fusion, 2020, 54: 99-118. DOI: 10.1016/j.inffus.2019.07.011
    [17]
    YAN H, YU X, et al. Single image depth estimation with normal guided scale invariant deep convolutional fields[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 29(1): 80-92.
    [18]
    LIU Y, CHEN X, WARD R, et al. Image fusion with convolutional sparse representation[J]. IEEE Signal Processing Letters, 2016, 23(12): 1882-1886. DOI: 10.1109/LSP.2016.2618776
    [19]
    MA J Y, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109. DOI: 10.1016/j.inffus.2016.02.001
    [20]
    MA J Y, XU H, JIANG J, et al. DDcGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion [J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. DOI: 10.1109/TIP.2020.2977573
    • Related Articles

  • Related Articles

    [1]JIN Yongwei, LIU Chunlong. Sensitivity Analysis of Thermal Design Parameters of High-Sensitivity Optical Detection Module[J]. Infrared Technology , 2025, 47(2): 141-147.
    [2]HE Qi, ZHAO Hangbin, PENG Jun, SUN Dexin. Application of Multiple Sampling Technology in LWIR Hyperspectral Imaging System[J]. Infrared Technology , 2019, 41(5): 457-461.
    [3]CHEN Jie, WU Xiaodi, YANG Xing, SHAO Hui. Calculation of Spatial Heat Flux Based on Improved Monte Carlo Algorithm[J]. Infrared Technology , 2017, 39(1): 91-94.
    [4]CHENG Wen, LIJun-shan, YU Ning, FENG Fan. Monte Carlo Simulation of Infrared Radiation Through Smoke Screen[J]. Infrared Technology , 2010, 32(11): 672-675,680. DOI: 10.3969/j.issn.1001-8891.2010.11.013
    [5]WANG Yi-feng, MAO Jing-xiang, FAN Nai-hua. Computation of Effectiveness of Baffled Cold Shield with Monte Carlo Method and MATLAB[J]. Infrared Technology , 2010, 32(9): 513-516. DOI: 10.3969/j.issn.1001-8891.2010.09.005
    [6]RONG Bai-lian, TANG Li-bin, YANG Yan, ZHU Ying-feng, LI Quan-bao. Discrimination of Unknown Lithium Element for Analysis Alloy Sample of Silver and Copper by GDMS[J]. Infrared Technology , 2010, 32(3): 169-172. DOI: 10.3969/j.issn.1001-8891.2010.03.012
    [7]CHEN Bo-yang, CHEN Fan-sheng, CHEN Gui-lin, SUN Sheng-li. A New Method of Improving Spatial Resolution of Linear Matrix Scanner by Over Sample[J]. Infrared Technology , 2009, 31(7): 395-398,402. DOI: 10.3969/j.issn.1001-8891.2009.07.006
    [8]XU Liang, LIU Jian-guo, GAO Min-guang, LU Yi-huai, LIU Wen-qing, WEI Xiu-li, ZHANG Tian-shu, LIU Zhi-ming. Compare the Results Measuring Methane in Atmosphere with Open Path FTIR And with Point Sampling Instrument[J]. Infrared Technology , 2007, 29(4): 239-242. DOI: 10.3969/j.issn.1001-8891.2007.04.013
    [9]Optimized Method of Selecting Samples for modeling in NIR Spectral Analysis[J]. Infrared Technology , 2005, 27(1): 75-78. DOI: 10.3969/j.issn.1001-8891.2005.01.018
    [10]Infrared FPA Spatial Sampling Characteristics Analysis and Measurement[J]. Infrared Technology , 2004, 26(2): 54-56,59. DOI: 10.3969/j.issn.1001-8891.2004.02.014

  • Cited by

    Periodical cited type(9)

    1. 卢泉，黄粒峰，胡梦竹. 基于改进直方图均衡的SF6泄漏区域增强算法. 红外技术. 2024(04): 437-442 . 本站查看
    2. 满林林，孙毅，李淑梅. 基于激光成像检测的GIS开关站SF_6气体泄露检测方法. 电器工业. 2024(11): 23-26+41 .
    3. 张泽林，吴玲玲，陈靖，王谦，张博渊. 红外成像的重气泄漏实时定位方法. 西安工业大学学报. 2024(06): 754-763 .
    4. 宋晓燕，李强，张峰，韩菲，王超冉. 电力基坑多气体浓度检测系统的研究与设计. 微型电脑应用. 2024(12): 183-186 .
    5. 袁建华，陈广生，张天宇，黄淘，陈轩. 基于红外与可见光图像融合的GIS设备气体泄漏识别研究. 国外电子测量技术. 2024(12): 231-239 .
    6. 曹江涛，李泉成，班铭，刘继臻，姬晓飞. 基于红外光谱成像的危险气体泄漏检测技术综述. 科学技术与工程. 2023(19): 8050-8060 .
    7. 刘赫，赵天成，李嘉帅，杨代勇，袁小翠，许志浩. 基于三直方图均衡的SF_6红外图像对比度增强方法. 红外技术. 2023(10): 1118-1125 . 本站查看
    8. 张红星，眭霄翔，王海军，刘中华，陈怀东，张海峰. 凝汽器管道壁面泄漏流场数值模拟研究. 真空. 2023(06): 15-21 .
    9. 李强，张峰，宋晓燕，韩菲，梁纲. 基于数值分析的电力基坑气体分布检测分析. 微型电脑应用. 2023(11): 91-94 .

    Other cited types(5)

Catalog

    Get Citation
    PDF
    XML
    Article views PDF downloads Cited by(14)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return