Citation: | CHEN Chaoyang, JIANG Yuanyuan. Infrared and Visible Image Fusion Based on Deep Image Decomposition[J]. Infrared Technology , 2024, 46(12): 1362-1370. |
Infrared and visible light image fusion is an enhancement technique designed to create a fused image that retains the advantages of the source image. In this study, a depth image decomposition-based infrared and visible image fusion method is proposed. First, the source image is decomposed into the background feature map and detail feature map by the encoder; simultaneously, the saliency feature extraction module is introduced in the encoder to highlight the edge and texture features of the source image; subsequently, the fused image is obtained by the decoder. In the training process, a gradient coefficient penalty was applied to the visible image for regularized reconstruction to ensure texture consistency, and a loss function was designed for image decomposition and reconstruction to reduce the differences between the background feature maps and amplify the differences between the detail feature maps. The experimental results show that the method can generate fused images with rich details and bright targets. In addition, this method outperforms other comparative methods in terms of subjective and objective evaluations of the TNO and FLIR public datasets.
[1] |
TANG L F, YUAN J, MA J Y. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42. DOI: 10.1016/j.inffus.2021.12.004
|
[2] |
罗迪, 王从庆, 周勇军. 一种基于生成对抗网络与注意力机制的可见光和红外图像融合方法[J]. 红外技术, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
LUO Di, WANG Congqing, ZHOU Yongjun. A visible and infrared image fusion method based on generative adversarial networks and attention mechanism[J]. Infrared Technology, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
|
[3] |
ZHANG X. Deep learning-based multi-focus image fusion: a survey and a comparative study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(9): 4819-4838. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9428544
|
[4] |
MA Jiayi, ZHOU Yi. Infrared and visible image fusion via gradientlet filter[J]. Computer Vision and Image Understanding, 2020(197-198): 103016.
|
[5] |
LI G, LIN Y, QU X. An infrared and visible image fusion method based on multi-scale transformation and norm optimization[J]. Information Fusion, 2021, 71: 109-129. DOI: 10.1016/j.inffus.2021.02.008
|
[6] |
MA J Y, YU W, LIANG P W, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
|
[7] |
刘佳, 李登峰. 马氏距离与引导滤波加权的红外与可见光图像融合[J]. 红外技术, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
LIU Jia, LI Dengfeng. Infrared and visible light image fusion based on Mahalanobis distance and guided filter weighting[J]. Infrared Technology, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
|
[8] |
TANG L F, YUAN J, MA J Y, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83: 79-92. http://www.sciencedirect.com/science/article/pii/S156625352200032X?dgcid=rss_sd_all
|
[9] |
TANG L F, XIANG X Y, ZHANG H, et al. DIVFusion: darkness-free infrared and visible image fusion[J]. Information Fusion, 2023, 91: 477-493. DOI: 10.1016/j.inffus.2022.10.034
|
[10] |
YU F, JUN X W, Tariq D. Image fusion based on generative adversarial network consistent with perception[J]. Information Fusion, 2021, 72: 110-125. DOI: 10.1016/j.inffus.2021.02.019
|
[11] |
何乐, 李忠伟, 罗偲, 等. 基于空洞卷积与双注意力机制的红外与可见光图像融合[J]. 红外技术, 2023, 45(7): 732-738. http://hwjs.nvir.cn/article/id/66ccbb6b-ad75-418f-ae52-cc685713d91b
HE Le, LI Zhongwei, LUO Cai, et al. Infrared and visible image fusion based on cavity convolution and dual attention mechanism[J]. Infrared Technology, 2023, 45(7): 732-738. http://hwjs.nvir.cn/article/id/66ccbb6b-ad75-418f-ae52-cc685713d91b
|
[12] |
ZHANG Y, LIU Y, SUN P, et al. IFCNN: a general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. DOI: 10.1016/j.inffus.2019.07.011
|
[13] |
MA J Y, XU H, JIANG J, et al. DDcGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion [J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. http://www.xueshufan.com/publication/3011768656
|
[14] |
MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14. http://doc.paperpass.com/foreign/rgArti2020150972012.html
|
[15] |
Prabhakar K R, Srikar V S, Babu R V. DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]// IEEE International Conference on Computer Vision (ICCV), 2017: 4724-4732.
|
[1] | HUANG Feiqing, GUO Baofeng, YOU Jingyun, WU Zhilong, WANG Yiwei, WANG Qinglin. A Feature Extraction Method of Hyperspectral Image with Multi-Scale Convolutional Filters[J]. Infrared Technology , 2025, 47(6): 712-721. |
[2] | LI Ziqian, BAN Yanwameng, LIU Yun, HE Dong, DU Rucai. Visible and Infrared Image Matching Method Based on Multi-Scale Feature Point Extraction[J]. Infrared Technology , 2025, 47(3): 351-357. |
[3] | LI Qiuheng, DENG Hao, LIU Guihua, PANG Zhongxiang, TANG Xue, ZHAO Junqin, LU Mengyuan. Infrared and Visible Images Fusion Method Based on Multi-Scale Features and Multi-head Attention[J]. Infrared Technology , 2024, 46(7): 765-774. |
[4] | CHONG Fating, DONG Zhangyu, YANG Xuezhi, ZENG Qingwang. SAR and Multispectral Image Fusion Based on Dual-channel Multi-scale Feature Extraction and Attention[J]. Infrared Technology , 2024, 46(1): 61-73. |
[5] | ZHANG Hui, HAN Xinning, HAN Huili, CHANG Lihong. Two-scale Image Fusion of Visible and Infrared Images Based on Guided Filtering Decomposition[J]. Infrared Technology , 2023, 45(11): 1216-1222. |
[6] | CHEN Sijing, FU Zhitao, LI Ziqian, NIE Han, SONG Jiawen. A Visible and Infrared Image Fusion Algorithm Based on Adaptive Enhancement and Saliency Detection[J]. Infrared Technology , 2023, 45(9): 907-914. |
[7] | TIAN Lifan, YANG Shen, LIANG Jiaming, WU Jin. Infrared and Visible Image Fusion Based on SGWT and Multi-Saliency[J]. Infrared Technology , 2022, 44(7): 676-685. |
[8] | DING Jian, GAO Qingwei, LU Yixiang, SUN Dong. Infrared and Visible Image Fusion Algorithm Based on the Decomposition of Robust Principal Component Analysis and Latent Low Rank Representation[J]. Infrared Technology , 2022, 44(1): 1-8. |
[9] | QIAN Zhenlong, CHEN Bo. Image Fusion Algorithm Based on Thermal Radiation Information Retention[J]. Infrared Technology , 2021, 43(9): 861-868. |
[10] | BAO Daerhan, GAO Wenwei, YANG Jinying. Fusion Algorithm for Infrared Intensity and Polarization Images Using Hybrid l0l1 Layer Decomposition[J]. Infrared Technology , 2020, 42(7): 676-682,701. |