Citation: | CHEN Chaoyang, JIANG Yuanyuan. Infrared and Visible Image Fusion Based on Deep Image Decomposition[J]. Infrared Technology , 2024, 46(12): 1362-1370. |
Infrared and visible light image fusion is an enhancement technique designed to create a fused image that retains the advantages of the source image. In this study, a depth image decomposition-based infrared and visible image fusion method is proposed. First, the source image is decomposed into the background feature map and detail feature map by the encoder; simultaneously, the saliency feature extraction module is introduced in the encoder to highlight the edge and texture features of the source image; subsequently, the fused image is obtained by the decoder. In the training process, a gradient coefficient penalty was applied to the visible image for regularized reconstruction to ensure texture consistency, and a loss function was designed for image decomposition and reconstruction to reduce the differences between the background feature maps and amplify the differences between the detail feature maps. The experimental results show that the method can generate fused images with rich details and bright targets. In addition, this method outperforms other comparative methods in terms of subjective and objective evaluations of the TNO and FLIR public datasets.
[1] |
TANG L F, YUAN J, MA J Y. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42. DOI: 10.1016/j.inffus.2021.12.004
|
[2] |
罗迪, 王从庆, 周勇军. 一种基于生成对抗网络与注意力机制的可见光和红外图像融合方法[J]. 红外技术, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
LUO Di, WANG Congqing, ZHOU Yongjun. A visible and infrared image fusion method based on generative adversarial networks and attention mechanism[J]. Infrared Technology, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
|
[3] |
ZHANG X. Deep learning-based multi-focus image fusion: a survey and a comparative study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(9): 4819-4838. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9428544
|
[4] |
MA Jiayi, ZHOU Yi. Infrared and visible image fusion via gradientlet filter[J]. Computer Vision and Image Understanding, 2020(197-198): 103016.
|
[5] |
LI G, LIN Y, QU X. An infrared and visible image fusion method based on multi-scale transformation and norm optimization[J]. Information Fusion, 2021, 71: 109-129. DOI: 10.1016/j.inffus.2021.02.008
|
[6] |
MA J Y, YU W, LIANG P W, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
|
[7] |
刘佳, 李登峰. 马氏距离与引导滤波加权的红外与可见光图像融合[J]. 红外技术, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
LIU Jia, LI Dengfeng. Infrared and visible light image fusion based on Mahalanobis distance and guided filter weighting[J]. Infrared Technology, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
|
[8] |
TANG L F, YUAN J, MA J Y, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83: 79-92. http://www.sciencedirect.com/science/article/pii/S156625352200032X?dgcid=rss_sd_all
|
[9] |
TANG L F, XIANG X Y, ZHANG H, et al. DIVFusion: darkness-free infrared and visible image fusion[J]. Information Fusion, 2023, 91: 477-493. DOI: 10.1016/j.inffus.2022.10.034
|
[10] |
YU F, JUN X W, Tariq D. Image fusion based on generative adversarial network consistent with perception[J]. Information Fusion, 2021, 72: 110-125. DOI: 10.1016/j.inffus.2021.02.019
|
[11] |
何乐, 李忠伟, 罗偲, 等. 基于空洞卷积与双注意力机制的红外与可见光图像融合[J]. 红外技术, 2023, 45(7): 732-738. http://hwjs.nvir.cn/article/id/66ccbb6b-ad75-418f-ae52-cc685713d91b
HE Le, LI Zhongwei, LUO Cai, et al. Infrared and visible image fusion based on cavity convolution and dual attention mechanism[J]. Infrared Technology, 2023, 45(7): 732-738. http://hwjs.nvir.cn/article/id/66ccbb6b-ad75-418f-ae52-cc685713d91b
|
[12] |
ZHANG Y, LIU Y, SUN P, et al. IFCNN: a general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. DOI: 10.1016/j.inffus.2019.07.011
|
[13] |
MA J Y, XU H, JIANG J, et al. DDcGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion [J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. http://www.xueshufan.com/publication/3011768656
|
[14] |
MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14. http://doc.paperpass.com/foreign/rgArti2020150972012.html
|
[15] |
Prabhakar K R, Srikar V S, Babu R V. DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]// IEEE International Conference on Computer Vision (ICCV), 2017: 4724-4732.
|
[1] | AI Zhiwei, ZHANG Mufan, ZHU Hua, JI Jianbo, BAI Yuanzhong. Design of Adaptive Inversion Proportional-Integral-Derivative Control System for Fast-Steering Mirror[J]. Infrared Technology , 2024, 46(2): 144-149. |
[2] | LI Shuai, YANG Baoyu, LU Yan. Adaptive PID Control Method Based on Space Optical Mechanical Thermal Model[J]. Infrared Technology , 2021, 43(10): 934-939. |
[3] | TANG Changming, ZHONG Jianfeng, ZHONG Shuncong, CHEN Man, FU Xibin, HUANG Xuebin. Ultrasound Infrared Thermography Defect Recognition Based on Improved Adaptive Genetic Algorithm with Two-Dimensional Maximum Entropy[J]. Infrared Technology , 2020, 42(8): 801-808. |
[4] | HUANG Yu, ZHANG Baohui, WU Jie, CHEN Yingyan, JI Li, WU Xudong, YU Shikong. Adaptive Multipoint Calibration Non-uniformity Correction Algorithm[J]. Infrared Technology , 2020, 42(7): 637-643. |
[5] | LI Zun, SHEN Xiaomeng, MIAO Tongjun. Image Mosaic Based on Contract Threshold Adaptive SIFT Algorithm[J]. Infrared Technology , 2017, 39(10): 946-950. |
[6] | HAO Yu, WANG Xinsai, ZHANG Yanbo, LU Jianfang, HE Jing, LIU Yu. The Infrared Image Enhancement Algorithm Based on Adapted Scale Factor Retinex[J]. Infrared Technology , 2016, 38(10): 855-859. |
[7] | GAO Xiao-dan, WEI Wan-hua. An Adaptive Enhancement Algorithm Based on Gaussian Distribution for Infrared Image[J]. Infrared Technology , 2014, 36(5): 381-383. |
[8] | YU Hong-sheng, JIN Wei-qi. SIFT Key-points Self-adaptive Extraction Algorithm for Video Images[J]. Infrared Technology , 2013, (12): 768-772. |
[9] | LI Xu, ZHAO Wen-jie, YANG Kai-da. OTSU Applied in Image Segmentation Based on Small Targets Pre-detection[J]. Infrared Technology , 2013, (8): 492-496. |
[10] | YAO Qin-fen, SUI Xiu-bao. An Adaptive Contrast Enhancement Algorithm for Infrared Image[J]. Infrared Technology , 2009, 31(9): 541-544. DOI: 10.3969/j.issn.1001-8891.2009.09.011 |