Advanced Search
中文
Volume 46 Issue 3
Mar.  2024
Turn off MathJax
Article Contents
Abstract
References
Related Articles
MA Luyao, LUO Xiaoqing, ZHANG Zhancheng. Infrared and Visible Image Fusion Based on Information Bottleneck Siamese Autoencoder Network[J]. Infrared Technology , 2024, 46(3): 314-324.
Citation: MA Luyao, LUO Xiaoqing, ZHANG Zhancheng. Infrared and Visible Image Fusion Based on Information Bottleneck Siamese Autoencoder Network[J]. Infrared Technology , 2024, 46(3): 314-324.
shu

Infrared and Visible Image Fusion Based on Information Bottleneck Siamese Autoencoder Network

  • 1.

    School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China

  • 2.

    Institute of Advanced Technology, Jiangnan University, Wuxi 214122, China

  • 3.

    Jiangsu Laboratory of Pattern Recognition and Computational Intelligence, Wuxi 214122, China

  • 4.

    School of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou 215000, China

More Information
  • Received Date: November 23, 2022
  • Revised Date: December 29, 2022
    Graphical Abstract
    • Abstract
  • Infrared and visible image fusion methods have problems such as insufficient information extraction, feature decoupling, and low interpretability. In order to fully extract and fuse the effective information of the source image, this paper proposes an infrared and visible image fusion method based on information bottleneck siamese autoencoder network (DIBF: Double Information Bottleneck Fusion). This method realizes the disentanglement of complementary features and redundant features by constructing an information bottleneck module on the twin branch. The expression process of complementary information corresponds to the feature fitting process of the first half of the information bottleneck. The compression process of redundant features corresponds to the feature compression process in the second half of the information bottleneck. This method cleverly expresses information extraction and fusion in image fusion as an information bottleneck trade-off problem, and achieves fusion by finding the optimal expression of information. In the information bottleneck module, the network obtains the information weight map of the feature through training, and uses the mean feature to compress the redundant features according to the information weight map. This method promotes the expression of complementary information through the loss function, and the two parts of compression and expression are balanced and optimized simultaneously. In this process, redundant information and complementary information are also decoupled. In the fusion stage, the information weight map is applied in the fusion rules, which improves the information richness of the fused images. Through subjective and objective experiments on the standard TNO dataset, compared with traditional and recent fusion methods, the results show that the method in this paper can effectively fuse useful information in infrared and visible images, and achieved good results on both visual perception and quantitative indicators.
    • FullText(HTML)
    • References (29)
  • [1]
    张冬冬, 王春平, 付强. 深度学习框架下的红外与可见光图像融合算法综述[J]. 激光与红外, 2022, 52(9): 1288-1298. DOI: 10.3969/j.issn.1001-5078.2022.09.004

    ZHANG D D, WANG C P, FU Q. Overview of infrared and visible image fusion algorithms based on deep learning framework[J]. Laser & Infrared, 2022, 52(9): 1288-1298. DOI: 10.3969/j.issn.1001-5078.2022.09.004
    [2]
    MA J, MA Y, LI C. Infrared and visible image fusion methods and applications: a survey[J]. Information Fusion, 2019, 45: 153-178. DOI: 10.1016/j.inffus.2018.02.004
    [3]
    陈永, 张娇娇, 王镇. 多尺度密集连接注意力的红外与可见光图像融合[J]. 光学精密工程, 2022, 30(18): 2253-2266.

    CHEN Y, ZHANG J J, WANG Z. Infrared and visible image fusion based on multi-scale dense attention connection network[J]. Optics and Precision Engineering, 2022, 30(18): 2253-2266.
    [4]
    孙彬, 诸葛吴为, 高云翔, 等. 基于潜在低秩表示的红外和可见光图像融合[J]. 红外技术, 2022, 44(8): 853-862. http://hwjs.nvir.cn/article/id/7fc3a60d-61bb-454f-ad00-e925eeb54576

    SUN B, ZHUGE W W, GAO Y X, et al. Infrared and visible lmage fusion based on latent low-rank representation[J]. Infrared Technology, 2022, 44(8): 853-862. http://hwjs.nvir.cn/article/id/7fc3a60d-61bb-454f-ad00-e925eeb54576
    [5]
    杨孙运, 奚峥皓, 王汉东, 等. 基于NSCT和最小化-局部平均梯度的图像融合[J]. 红外技术, 2021, 43(1): 13-20. http://hwjs.nvir.cn/article/id/144252d1-978c-4c1e-85ad-e0b8d5e03bf6

    YANG S Y, XI Z H, WANG H D, et al. Image fusion based on NSCT and minimum-local mean gradient [J]. Infrared Technology, 2021, 43(1): 13-20. http://hwjs.nvir.cn/article/id/144252d1-978c-4c1e-85ad-e0b8d5e03bf6
    [6]
    刘智嘉, 贾鹏, 夏寅辉. 基于红外与可见光图像融合技术发展与性能评价[J]. 激光与红外, 2019, 49(5): 123-130. https://www.cnki.com.cn/Article/CJFDTOTAL-JGHW201905022.htm

    LIU Z J, JIA P, XIA Y H, et al. Development and performance evaluation of infrared and visual image fusion technology[J]. Laser & Infrared, 2019, 49(5): 123-130. https://www.cnki.com.cn/Article/CJFDTOTAL-JGHW201905022.htm
    [7]
    Lee H Y, Tseng H Y, Mao Q, et al. Drit++: Diverse image-to-image translation via disentangled representations[J]. International Journal of Computer Vision, 2020, 128(10): 2402-2417.
    [8]
    马梁, 苟于涛, 雷涛, 等. 基于多尺度特征融合的遥感图像小目标检测[J]. 光电工程, 2022, 49(4): 49-65. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202204005.htm

    MA L, GOU Y T, LEI T, et al. Small object detection based on multi-scale feature fusion using remote sensing images[J]. Opto-Electronic Engineering, 2022, 49(4): 49-65. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202204005.htm
    [9]
    雷大江, 杜加浩, 张莉萍, 等. 联合多流融合和多尺度学习的卷积神经网络遥感图像融合方法[J]. 电子与信息学报, 2022, 44(1): 237-244. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX202201025.htm

    LEI D J, DU J H, ZHANG L P, et al. Multi-stream architecture and multi-scale convolutional neural network for remote sensing image fusion[J]. Journal of Electronics & Information Technology, 2022, 44(1): 237-244. https://www.cnki.com.cn/Article/CJFDTOTAL-DZYX202201025.htm
    [10]
    李明, 刘帆, 李婧芝. 结合卷积注意模块与卷积自编码器的细节注入遥感图像融合[J]. 光子学报, 2022, 51(6): 406-418. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB202206038.htm

    LI M, LIU F, LI J Z. Combining convolutional attention module and convolutional autoencoder for detail injection remote sensing image fusion[J]. Acta Photonica Sinica, 2022, 51(6): 406-418. https://www.cnki.com.cn/Article/CJFDTOTAL-GZXB202206038.htm
    [11]
    刘博, 韩广良, 罗惠元. 基于多尺度细节的孪生卷积神经网络图像融合算法[J]. 液晶与显示, 2021, 36(9): 1283-1293. https://www.cnki.com.cn/Article/CJFDTOTAL-YJYS202109009.htm

    LIU B, HAN G L, LUO H Y. Image fusion algorithm based on multi-scale detail siamese convolutional neural network[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(9): 1283-1293. https://www.cnki.com.cn/Article/CJFDTOTAL-YJYS202109009.htm
    [12]
    Krishna V A, Reddy A A, Nagajyothi D. Signature recognition using siamese neural networks[C]//IEEE International Conference on Mobile Networks and Wireless Communications (ICMNWC), 2021: 1-4.
    [13]
    LI H, WU X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2614-2623.
    [14]
    LI H, WU X J, Durrani T. NestFuse: An infrared and visible image fusion architecture based on nest connection and spatial/channel attention models[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69(12): 9645-9656. DOI: 10.1109/TIM.2020.3005230
    [15]
    LU B, CHEN J C, Chellappa R. Unsupervised domain-specific deblurring via disentangled representations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 10225-10234.
    [16]
    WANG G, HAN H, SHAN S, et al. Cross-domain face presentation attack detection via multi-domain disentangled representation learning[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6678-6687.
    [17]
    文载道, 王佳蕊, 王小旭, 等. 解耦表征学习综述[J]. 自动化学报, 2022, 48(2): 351-374. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO202202003.htm

    WEN Z D, WANG J R, WANG X X, et al. A review of disentangled representation learning[J]. Acta Automatica Sinica, 2022, 48(2): 351-374. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO202202003.htm
    [18]
    ZHAO Z, XU S, ZHANG C, et al. DIDFuse: Deep image decomposition for infrared and visible image fusion[J]. arXiv preprint arXiv: 2003.09210, 2020.
    [19]
    XU H, WANG X, MA J. DRF: Disentangled representation for visible and infrared image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-13.
    [20]
    XU H, GONG M, TIAN X, et al. CUFD: An encoder–decoder network for visible and infrared image fusion based on common and unique feature decomposition[J]. Computer Vision and Image Understanding, 2022, 218: 103407. DOI: 10.1016/j.cviu.2022.103407
    [21]
    Tishby N, Pereira F C, Bialek W. The information bottleneck method[J]. arXiv preprint physics/0004057, 2000.
    [22]
    Tishby N, Zaslavsky N. Deep learning and the information bottleneck principle[C]// IEEE Information Theory Workshop (ITW). IEEE, 2015: 1-5.
    [23]
    Shwartz-Ziv R, Tishby N. Opening the black box of deep neural networks via information[J]. arXiv preprint arXiv: 1703.00810, 2017.
    [24]
    Alemi A A, Fischer I, Dillon J V, et al. Deep variational information bottleneck[J]. arXiv preprint arXiv: 1612.00410, 2016.
    [25]
    Tishby N, Zaslavsky N. Deep learning and the information bottleneck principle[C]//IEEE Information Theory Workshop (ITW). IEEE, 2015: 1-5.
    [26]
    ZHANG Y, LIU Y, SUN P, et al. IFCNN: A general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. DOI: 10.1016/j.inffus.2019.07.011
    [27]
    MA J, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109. DOI: 10.1016/j.inffus.2016.02.001
    [28]
    ZHANG H, MA J. SDNet: A versatile squeeze-and-decomposition network for real-time image fusion[J]. International Journal of Computer Vision, 2021, 129(10): 2761-2785. DOI: 10.1007/s11263-021-01501-8
    [29]
    LIU Y, LIU S, WANG Z. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164. DOI: 10.1016/j.inffus.2014.09.004
    • Related Articles

  • Related Articles

    [1]ZHOU Fang, WANG Guanghua, ZHOU Yunhong, DUAN Qian, XIE Hongyi, YANG Weiping, JIN Jingyi, SUN Weijie, ZUO Lina, SHI Mei. The Impact of Curing Technology on Reliability of Glass Packaging of OLED Micro Displays[J]. Infrared Technology , 2025, 47(6): 779-784.
    [2]YANG Qiming, QIAN Fuli, GOU Guoru, WANG Tilu, LU Chaoyu, ZHOU Yunhong, DUAN Qian, YU Xiaohui, DUAN Yu, WANG Guanhua, YANG Wenyun. Research on Capping Layer for Top-emitting White OLED Micro-display Performance[J]. Infrared Technology , 2023, 45(3): 303-307.
    [3]QIN Guohui, YU Xiaohui, QIAN Fuli, DUAN Yu, YANG Qiming, GOU Guoru. Improvement of Color Purity of Organic Monochromatic Green Top-emitting Micro-display Devices by Using Optical Microcavity[J]. Infrared Technology , 2022, 44(7): 652-658.
    [4]YANG Qiming, GAO Sibo, WANG Can, DUAN Liangfei, QIAN Fuli, DUAN Qian, ZHANG Jie, WANG Guanghua, LU Chaoyu, DUAN Yu. Study on the Effects of Yb: Ag Alloy Cathode on the Photoelectric Performance of the Top Emitting White Organic Light-emitting Devices[J]. Infrared Technology , 2021, 43(12): 1207-1211.
    [5]WANG Guanghua, ZHOU Fang, CHEN Xuemei, GAO Sibo, ZHANG Jie, DUAN Yu, DUAN Liangfei, QIAN Fuli, YANG Qiming, WU Yanming, ZHAO Mengling, JI Huaxia. Preparation and Performance of Top-Emitting Organic Green Light Emitting Devices[J]. Infrared Technology , 2020, 42(9): 817-822.
    [6]DUAN Liangfei, DUAN Yu, WANG Guanghua, ZHANG Xiaodan, DENG Rongbin, JI Huaxia. Research on High-Performance Composite Anode of Top-emitting OLED Devices[J]. Infrared Technology , 2017, 39(5): 457-462.
    [7]DUAN Yu, ZHANG Xiao-dan, SUN Hao, ZHU Ya-an, WANG Guang-hua, SONG Li-yuan, YU Xiao-hui, WAN Rui-min, JI Hua-xia, LI Ya-wen. Fabrication of High Brightness Top-emitting Green OLED Micro-display[J]. Infrared Technology , 2015, (12): 1022-1026.
    [8]KONG Ling-de, FANG Hui, WEI Hong, LIU Li, ZHOU Ruan-sheng, TIE Xiao-ying, YANG Wen-yun, JI Rong-bin. The Photoelectrical Properties of VOxFilm by As-deposited SiOxDielectric Layer[J]. Infrared Technology , 2015, (4): 319-322.
    [9]LI Yan-dong, YANG Jun-yan, LI Ya-wen, ZHOU Qin. Application of AM-OLED Microdisplay in Infrared Display Systems[J]. Infrared Technology , 2012, 34(4): 200-204,223. DOI: 10.3969/j.issn.1001-8891.2012.04.004
    [10]YU Lian-jie, SHI Yan-li, DENG Gong-rong, LI Xiong-jun, YANG Li-li, HE Wen-jin. The Research on Photoelectrical Properties of Amorphous HgTe Thin Films[J]. Infrared Technology , 2011, 33(4): 190-194. DOI: 10.3969/j.issn.1001-8891.2011.04.002

  • Cited by

    Periodical cited type(2)

    1. 曹志群，项高鹏，胡临天，廖梓宏. 基于近红外技术的非接触式酒精度检测系统. 激光与红外. 2025(03): 408-413 .
    2. 张肖，朱铧丞，杨阳，李兴兴. 基于随机森林算法的酒精浓度在线测量系统. 真空电子技术. 2023(02): 80-86 .

    Other cited types(0)

Catalog

    Get Citation
    PDF
    XML
    Article views (117) PDF downloads (30) Cited by(2)
    Related

    Copyright © 《Infrared Technology》Editorial Office滇ICP备12000354号-3

    Supported by: Beijing Renhe Information Technology Co., Ltd.  Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return