Citation: | CHEN Chaoyang, JIANG Yuanyuan. Infrared and Visible Image Fusion Based on Deep Image Decomposition[J]. Infrared Technology , 2024, 46(12): 1362-1370. |
Infrared and visible light image fusion is an enhancement technique designed to create a fused image that retains the advantages of the source image. In this study, a depth image decomposition-based infrared and visible image fusion method is proposed. First, the source image is decomposed into the background feature map and detail feature map by the encoder; simultaneously, the saliency feature extraction module is introduced in the encoder to highlight the edge and texture features of the source image; subsequently, the fused image is obtained by the decoder. In the training process, a gradient coefficient penalty was applied to the visible image for regularized reconstruction to ensure texture consistency, and a loss function was designed for image decomposition and reconstruction to reduce the differences between the background feature maps and amplify the differences between the detail feature maps. The experimental results show that the method can generate fused images with rich details and bright targets. In addition, this method outperforms other comparative methods in terms of subjective and objective evaluations of the TNO and FLIR public datasets.
[1] |
TANG L F, YUAN J, MA J Y. Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82: 28-42. DOI: 10.1016/j.inffus.2021.12.004
|
[2] |
罗迪, 王从庆, 周勇军. 一种基于生成对抗网络与注意力机制的可见光和红外图像融合方法[J]. 红外技术, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
LUO Di, WANG Congqing, ZHOU Yongjun. A visible and infrared image fusion method based on generative adversarial networks and attention mechanism[J]. Infrared Technology, 2021, 43(6): 566-574. http://hwjs.nvir.cn/article/id/3403109e-d8d7-45ed-904f-eb4bc246275a
|
[3] |
ZHANG X. Deep learning-based multi-focus image fusion: a survey and a comparative study[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(9): 4819-4838. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9428544
|
[4] |
MA Jiayi, ZHOU Yi. Infrared and visible image fusion via gradientlet filter[J]. Computer Vision and Image Understanding, 2020(197-198): 103016.
|
[5] |
LI G, LIN Y, QU X. An infrared and visible image fusion method based on multi-scale transformation and norm optimization[J]. Information Fusion, 2021, 71: 109-129. DOI: 10.1016/j.inffus.2021.02.008
|
[6] |
MA J Y, YU W, LIANG P W, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26. DOI: 10.1016/j.inffus.2018.09.004
|
[7] |
刘佳, 李登峰. 马氏距离与引导滤波加权的红外与可见光图像融合[J]. 红外技术, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
LIU Jia, LI Dengfeng. Infrared and visible light image fusion based on Mahalanobis distance and guided filter weighting[J]. Infrared Technology, 2021, 43(2): 162-169. http://hwjs.nvir.cn/article/id/56484763-c7b0-4273-a087-8d672e8aba9a
|
[8] |
TANG L F, YUAN J, MA J Y, et al. PIAFusion: a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022, 83: 79-92. http://www.sciencedirect.com/science/article/pii/S156625352200032X?dgcid=rss_sd_all
|
[9] |
TANG L F, XIANG X Y, ZHANG H, et al. DIVFusion: darkness-free infrared and visible image fusion[J]. Information Fusion, 2023, 91: 477-493. DOI: 10.1016/j.inffus.2022.10.034
|
[10] |
YU F, JUN X W, Tariq D. Image fusion based on generative adversarial network consistent with perception[J]. Information Fusion, 2021, 72: 110-125. DOI: 10.1016/j.inffus.2021.02.019
|
[11] |
何乐, 李忠伟, 罗偲, 等. 基于空洞卷积与双注意力机制的红外与可见光图像融合[J]. 红外技术, 2023, 45(7): 732-738. http://hwjs.nvir.cn/article/id/66ccbb6b-ad75-418f-ae52-cc685713d91b
HE Le, LI Zhongwei, LUO Cai, et al. Infrared and visible image fusion based on cavity convolution and dual attention mechanism[J]. Infrared Technology, 2023, 45(7): 732-738. http://hwjs.nvir.cn/article/id/66ccbb6b-ad75-418f-ae52-cc685713d91b
|
[12] |
ZHANG Y, LIU Y, SUN P, et al. IFCNN: a general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. DOI: 10.1016/j.inffus.2019.07.011
|
[13] |
MA J Y, XU H, JIANG J, et al. DDcGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion [J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995. http://www.xueshufan.com/publication/3011768656
|
[14] |
MA J, ZHANG H, SHAO Z, et al. GANMcC: a generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14. http://doc.paperpass.com/foreign/rgArti2020150972012.html
|
[15] |
Prabhakar K R, Srikar V S, Babu R V. DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]// IEEE International Conference on Computer Vision (ICCV), 2017: 4724-4732.
|
[1] | DAI Zikuo, SHI Kejian, SONG Shida, LIU Yang, XU Yan. Reliability Image Recognition Method for High Temperature Operation of Power Stabilizer in Medium and Low Voltage Grids Based on Infrared Imaging[J]. Infrared Technology , 2023, 45(12): 1351-1357. |
[2] | YUAN Xilin, ZHANG Baohui, ZHANG Qian, HE Ming, ZHOU Jinjie, LIAN Cheng, YUE Jiang. Infrared Images with Super-resolution Based on Deep Convolutional Neural Network[J]. Infrared Technology , 2023, 45(5): 498-505. |
[3] | CAO Yutong, HUAN Kewei, XUE Chao, HAN Fengdi, LI Xiangyang, CHEN Xiao. Infrared and Visible Image Fusion Based on CNN with NSCT[J]. Infrared Technology , 2023, 45(4): 378-385. |
[4] | XIONG Yu, SHAN Deming, YAO Yu, ZHANG Yu. Hyperspectral Image Hybrid Convolution Classification under Multi-Feature Fusion[J]. Infrared Technology , 2022, 44(1): 9-20. |
[5] | DONG Anyong, DU Qingzhi, SU Bin, ZHAO Wenbo, YU Wen. Infrared and Visible Image Fusion Based on Convolutional Neural Network[J]. Infrared Technology , 2020, 42(7): 660-669. |
[6] | LIAO Xiaohua, CHEN Niannian, JIANG Yong, QI Shifeng. Infrared Image Super-resolution Using Improved Convolutional Neural Network[J]. Infrared Technology , 2020, 42(1): 75-80. |
[7] | GAO Jun, JING Yiguo. A Fully Convoluted Neural Network-based Cloud Detection Method for Satellite Remote Sensing Images[J]. Infrared Technology , 2019, 41(7): 607-615. |
[8] | Document Image Classification Based on Improved Local Binary Patterns[J]. Infrared Technology , 2014, (10): 827-831. |
[9] | XU Xiang-jun, WANG Sheng-peng, JI Qing-chun, LIU Dong-fang, QIAN Wei-dong, YU Jie, YAN Ya-jing. Insulator Infrared Image Recognition Method Based on Gaussian Scale-space and GHT[J]. Infrared Technology , 2014, (7): 596-599. |
[10] | The Study of Feature Extraction Used to Recognize Incomplete Image for Imaging Fuze[J]. Infrared Technology , 2001, 23(5): 20-23,28. DOI: 10.3969/j.issn.1001-8891.2001.05.007 |
1. |
聂磊,武丽丽,黄一凡,刘梦然,刘江林. 基于红外图像分析的TSV内部缺陷识别方法研究. 仪表技术与传感器. 2023(01): 38-43 .
![]() | |
2. |
刘凯. 基于红外图像识别技术的道路与桥梁故障诊断. 计算技术与自动化. 2022(03): 105-110 .
![]() | |
3. |
沈九美,邱建林. 基于光散射的图形元素视觉整合系统. 激光杂志. 2022(10): 62-66 .
![]() |